Like most people before me, I would give the advice to start with preconstructed decks. Until recently WOTC delivered four or five commander decks a year, but nowadays each standard legal set comes with two commander decks with it. Those decks don't have as much new cards in them as the mentioned 'bigger commander sets' but they are cheaper and have enough of important staples like
command tower,
sol ring, or
arcane signet in them. And these decks play nice against each other.
If you are planning on building decks yourselfs, investing in mana producers as lands and manarocks is the best start. As said there are quite useful ones in the precons, but you could buy a quite a few other useful lands as well. I like lands like
reliquary tower,
rogue's passage because they have an useful ability and are cheap as well. And for other information on lands you can check this topic on deckstats as well:
https://deckstats.net/forum/index.php/topic,55692.msg163560.html#msg163560.
If you're in to it, there are podcast and youtube channels focused on (budget) commander, like Commander's Quarters. The aforementioned site EDHREC.com has ways to select the cheapest builds of each commander and have article series on budget building.
Me and a couple of friends have started to build decks with only commons and uncommons now there are so many legendary creatures printed at uncommon. So there are quite a lot of ways to get started with building, collecting and playing, depending on how much time and money you are willing to spend.