Also living in Worcester now and from what I can gather the RU, basketball and Hockey teams are well funded and are on the rise. Where as the football team has been fighting with the council for land to build a stadium on since the last ground was demolished for housing (don't know why without digging into it, so they may never have owned it)
Around the time Lee Hughes (the same) was there manager they got in financial issues and had no ground so they elected to be relegated down further in the leagues to avoid alot more financial fines etc.
So not too dissimilar to other football teams where it's not the go to sporting event for that town/area.
At least that's the story from fans I know at work and also throw into it the lower league clubs like Kidderminster etc they are also competing with.