Music Theory

While I was listening to some of my old vinyl, I stumbled across a fairly depressing realization.  I think that I'm further back with my music theory than I've been giving myself credit for.  I feel like I spend a lot of time practicing scales, but I don't spend nearly enough time improvising, and I spend almost no time at all playing with other people (or other live music).  Some of the stuff that people do with blues/jazz guitar is pretty far above what seems possible for me.  There are a lot of areas to explore and I feel like I don't know where to start.  I know some basic music theory concepts, but just listening to what can be done makes me wonder what it's going to take to get there.  A lot of practice?  Being around other people that practice a lot?  Maybe just listening to more music can help.