I also do Aikido, it's great.
The first thing you learn is how to fall properly, which - I believe - has already saved me once when I was going downhill on my bike and the handlebar got stuck, sending me flying - and I knew the feeling enough by then to turn and take the impact on the soft part of my hip and only scraping my head lightly. Had I not changed my posture, I might have gotten serious brain damage or died. This ability alone is enough to study martial arts!
As I'm a 50kg woman it also helps that it's not about physical power at all. If one needs force it's done wrong. Some techniques are better for bigger opponents and some are better for smaller ones, but you can send them all flying. There are no competitions because people would get hurt; it's scary how easy it is to break stuff. There is a lot of trust between training partners because you trust them not to break your body. (I'm not talking about pain here, that happens a lot.. and you learn to feel the level that you can take without injury.)
My dojo has a lot of blackbelts, which is important, because you can learn most effectively when an experienced teacher is there. Also, I'm finally at a level where I can take some moves and that makes me a better training partner for others.
Actually, there is a lot going on spiritually. The old master (over 80, 7th dan, it's amazing they gave this to a non-japanese person, but then he is amazing) talks a lot about spirit, and how it influences the techniques.
Techniques are hard to convey when you have the rough stuff down - I believe these mental things are brain patterns that allow to understand and control the body better than just saying "put your arm here" - (but that might just be me). The old master used to be a university math professor, and there are quite a few scientists around, which gives their words more weight in my eyes (we're a university club) - if it didn't work they wouldn't say it. Sometimes, things just work, and I'll keep trying.
We also do ken techniques, because you can use the idea behind them without a sword. Training forces me to concentrate really hard, to *be* there, because if not I can get hit. The mental state is great. I am centered, and directing my power in one direction (ideally

), and controlling the space around me (or there will be bruises..

) - I can understand how this can be meditation.
The downside is that Aikido takes years and years to learn, however. But even if you are over 80 like the old master, you will still get stronger - he gets stronger every year, it's insane.
Go do it, for a few disciplines fitness does not matter, or you will get fit soon enough, and you will be stronger not just in body. I'm a small girl, if I can do it, so can you.
TL;DR: GO DO IT!