They're probably on the other island (the small one near the normal one). And a bunch of them died in the season 3 finale. The only one who just disappeared is Richard, because the actor joined a new show, Cane. Although, I think Cane just got cancelled so he might be back :)

Yeah good point. I was wondering if some were in other hatches just holing up...and you'd think there'd still be others on the other island.

They went off to The Temple.

RICHARD: We're going to the radio tower?

BEN: Not we, Richard, me. You're going to take everyone to the temple as planned.

RICHARD: Now might not be the best time to go tromping off on your own.

BEN: Oh really and why not?

RICHARD: Because people are asking questions, Ben. About leaving home, about what happened to Locke, about Jacob. Not to mention the rapidly spreading rumour that everyone that went down to the beach is dead.

BEN: They're not all dead.