I think for JLPT1, it gets tested both ways: in isolation and in context.

I don't know how to answer your question without contaminating your 'no study' experiment.  I guess it depends how strictly you want to adhere to your policy.

------------------------

If it is loose, you can look here for links to sample questions:
http://www.jlpt.jp/e/about/sample.html

If it is strict, you will find out the answer to your question when you take it for real...

nest0r wrote:

Does the JLPT test kanji in isolation somehow, or when they say 'know 2000 kanji' do they just mean integrated into the language? If it's the former, that might make my 'no studying' experiment slightly more difficult since there'd be more of a standardized, specific system, I imagine.

They mean integrated into the language.

Tobberoth wrote:

You aren't tested on kanji in isolation so as long as you know all the words deemed necessary for JLPT2 AND you know which kanji those words are made up from, you should be all clear. You get tested in two ways if I don't remember incorrectly:
1. You get a word and 4 kanji to choose from, pick the correctly written kanji (If you have passed RtK1, you should get 100% correct on this I suppose).
2. You get a jukugo and you have to pick from 4 hiragana readings. As long as you know the word in question, this is really simple of course. If you don't know the word, it depends on how good you are with the readings of the individual kanji... i wouldn't recommend that solution.

For JLPT1 there are also questions where you are given a word written in hiragana and you have to pick a word (also written in hiragana) such that both words share a character when written in kanji. The options may all use kanji that differ only in their hen radical, for example.