I think the main problem is that you try to send the whole game state every interval.
A better way would be to split the sync into events and states. Events are important things like a spell that gets casted or damage that is dealt. Events need to get send over TCP because such packages are reliable and sequenced. States are components like the position and rotation of entities. States are sent over UDP because the sequence or packet loss does not matter here, the UDP client sends every interval. Depending on the maximum player count you can use multicasting for the UDP client.
Example:
// Sent when an event was requested by a client
public struct ServerEvent
{
public int affectedEntityId; // The entity that is affected by this event
public int receiveDamageAmount;
public int castedSpellId;
public int castedSpellTargetEntityId;
// ...
}
// Sent every interval (the Source engine uses 15ms66.6ms)
public struct ServerState
{
public int affectedEntityId;
public float xPosition, yPosition;
public float angle;
// ...
}
Because you do not send the whole state of your game, you will need to identify content in your scene. Spells and items the players collect need to be synced so that when a client requests the casting of a spell, every other peer knows what happened. This is easily managable with IDs. Every time new content is added to the scene a new event gets spread over the network. Containing the resource itself and an ID you can easily address specific entities in the game. On the server, these resources can be stored in a dictionary that has a sample instance of every object. This gives you not only a centralized management of ingame content and versioning, you can also add new content to your project, without even updating the client application.
// Sent when a resource is instantiated
public struct ServerSpawn
{
public object resourceState; // The serialized state
public ResourceType type; // The type, identifiable with an enum
public int resourceNetworkId;
}
To conclude, my approach on the problem would solve this issue by centralizing the data of your game and splitting the communication in this client/server model into events and states.