So I just failed N3, which wasn't a big surprise. My big frustration is that the test is only offered once a year where I live. I use Anki to study every day, and if it was offered here in June I'd sign up now just for the motivator to keep me studying and seeing improvement. I self-study, so I don't have the pressure and direction that comes with, say, being a student in a university.

I've heard people talk about online tests in the past, but never paid much attention to them. I'm looking for something I can take today and then again in 3 or 6 months just to sort of keep me honest about whether my studying is helping. I'm still planning to retake N3 next December. But that's so far out that it doesn't quite feel "real" yet.

Try this out: http://www.j-cat.org/en/

It sounds like what you're looking for. It's similar to JLPT in a way, except the questions get progressively more difficult based on how well you're doing. They also say that you're only allowed to take it once every 6 months I think it was, because it's supposed to gauge progress instead of trying to get the highest score by retaking it often.

And post in the J-CAT thread if you do.

Thanks guys - this is indeed exactly what I was looking for. I will try to take it soon and add a calendar item to take it again in 6 months. I've posted this elsewhere but I've alloted about 6 months to finish my N3 kanji book and my N4 grammar book. So getting a before and after snapshot should be helpful to me.

This is cool.  I like that I can take it in the comfort of my home on my own schedule instead of one fixed day per year.  I'll probably give it a try as I've never properly tested my level.  I prefer testing my level by what I am actually able to do with Japanese, but still I'm curious to see which areas are lacking and need improvement.