Lambeth North gets a bit of commuter traffic; and, especially, lots of tourists - both inbound for the IWM (at weekends particularly) and (more importantly) visitors (including from overseas) who're staying at hotels in the area. It's a bit like Russell Square in that it seems to those who don't know the area to be a bit in the middle of nowhere, slightly away from main centres of activity, not far from other stations, and with no interchange or anything to make it more useful - yet plenty of people staying in the area use it. In fact, Lambeth North (especially because it's got a very small entrance) was - in non-COVID times - a good place to avoid in the morning just after the rush hour: it was sometimes crowded with groups of foreign tourists (either starting a day out in London, or with bags, off to the airport or whatever) trying to figure out how to use the ticket machines. Since the ticket office there closed, I've seen locals who need to get or charge up a ticket before travelling give up on account of the confused tourists taking forever, and walk on to Waterloo to start their journey. So, it seems to me to be busy at times - for its above-ground size at least. Admittedly, its figures are probably low for quite a central station - and in the days when less-used stations were closed on Boxing Day and Bank Holidays, Lambeth North was one that closed.
As for Regents Park - well, at weekends you often see people using it to get to ... um ... Regents Park. (As I do sometimes if I'm not on my bike.) I expect the demographic in the buildings around it is - these days - mostly of people who don't travel by tube! (Maybe that was different a century ago.) At peak times, if you're going to or from somewhere between there and Oxford Circus, and using the Bakerloo, you'd be well-advised to use Regents Park, even if your origin/destination was a bit closer to Oxford Circus, to avoid the excessive crush at the latter.