If you're asking about frustum culling for rendering performance, then yes, it would benefit you in a multiplayer game for the same reasons it would in a single-player game. There's no difference in how rendering works. This type of culling would be done on the client; there's no point in doing it on the server.
If you're asking about frustum culling for networking, i.e. the server only sends player X position updates etc. about players Y, Z if player X can see Y and Z, then I can see some benefits and some drawbacks. Benefits: it reduces the network bandwidth, which is beneficial both for the server and for players with slow connections (as long as the players are relatively separated; if you get all the players in one room, it doesn't help). It also helps prevent cheating with wallhacks, since if you don't send updates about other players' positions, there's less opportunity for a hacker to gain an unfair advantage by knowing where they are.
On the drawbacks side, there could be some lag where you don't see a player even though they should be in your field of view, because the server hasn't been updated with your latest movements yet. To counter this you probably wouldn't want to do actual frustum culling, as you can rotate the camera very quickly and see players behind you. You'd want to just do position-based occlusion culling, or culling based on the level layout (keeping track of which room the player is in and which rooms can be seen from it - kind of like an old-fashioned PVS system). You'd also want to build some slack into the system so that if a player moves quickly they don't outrun the server's view of which players they can see.