Connecting to existing VM with OS

Just to ensure..

You have:
1) VM with Rally
2) VM with OpenStack

You would like to run Rally against this OpenStack?

Yes, that is what I am trying to do

So you should create Rally deployment (that will just present existing one).

Lets assume that admin enpdoint is next: username: admin, password: admin, tenant: admin, auth_url: some_url

so you should create json file (e.g. mycloud.json) with next lines:
  {
    "name": "DummyEngine",
    "endpoint": {
      "auth_url": "some_url",
      "username": "admin",
      "password": "admin",
      "tenant_name": "admin"
    }
  }

So now you should run:
  # rally deployment create --file mycloud.json --name mycloud --use

(If you specify --use it will pass as default deployment uuid, uuid of this created deployment)

So now you should go to horizon and find UUID of cirros images (or upload new images)
and write task input config (task.json):
  {
  "NovaServers.boot_and_delete_server": [
    {"args": {"flavor_id": 42, "image_id": "<YOUR_IMAGE_ID>"},
     "config": {"times": 10, "active_users": 2, "tenants": 2, "users_per_tenant": 3}
     }
   ]
  }

and run task:

rally -v task start task.json

This will run task against your cloud. In task there is only one benchmark scneario, that will do next things:
1) Create 2 TMP tenants
2) Create 3 TMP users in each of these tenants
3) Call this method: NovaServers.boot_and_delete_server 10 times every time using 2 simultaneously working users (that are for every call randomly chosen from temp users)

(btw I am hard working around Rally wiki now, so I made recently some changes on main page, probably it will help you a bit https://wiki.openstack.org/wiki/Rally)