There is no hard-and-fast rule on this, although it is recommended to invite people (including friends), if you can afford to. If your budget is tight, you may invite only the limited number of people as you have mentioned. Marriage is still valid even if you did not throw a big feast. Walimah does not have to be an expensive affair; it can even be limited to distributing some drinks and dates, if that is what a person can afford to; however, if someone can afford, he should do better.

In conclusion, you may make a walimah feast inviting a limited number of people-if that is what you can afford to.