Is it possible to install 2.0 to my host in a different directory, and play with it while having a 1.1.6 site, and if so is it possible to transfer all the user data posts etc later? without upgrading the 1.1.6?
Yes it sure is. As long as you set a different database for the new forum so it doesnt conflict with the 1.1.6 database!
Its not currently possible to transfer the posts over.
As long as you specify a different table prefix and/or database prefix, then there won't be any conflict in the database.