next »

[Marvin Fröhlich]

As mentioned in the 1.0 features thread we need a good net-code API. I have never coded or used a net-API. But I would say, it should care about low GC and net-traffic.

So it should NOT serialize whole objects and transmit them between server and client, but do the following:

• Mange a "registry" of items referred by a simple id.
• Provide the possibility to request an item-id binding.
• Provide methods to transmit a set of primitive parameters in combination with an item-id (asynchronously)
• Receive events by id/parameters
• Client and server would match the ID to the registered object and execute a method passing the parameters.

This way the API would remain slim and convenient at the same time and would not produce any GC by itself and the least net-traffic.

What do you think? Is there a good one out there, that we could use?

Marvin

[Mathias 'cylab' Henze]

We should have a look at Apache Mina (http://mina.apache.org/index.html) and JGN (https://javagamenetworking.dev.java.net/). The first may be the most versatile java networking library while the latter already has jME and jME physics binding, which probably could be adapted to xith (or at least can provide insights regarding implementation details and possible catch ups). At some time, we should also have a look at Project Darkstar (http://www.projectdarkstar.com/index.php?option=com_content&task=view&id=19&Itemid=37)

['n ddrylliog]

There is some interesting net code in Stratagem indeed (well you'll later tell me if it was interesting or not ^^)

But pretty much everything in Stratagem would need to be rewritten, and I'm writing some network for another (commercial) project, network code which can be open-sourced

[Mathias 'cylab' Henze]

There is some interesting net code in Stratagem indeed (well you'll later tell me if it was interesting or not ^^)

But pretty much everything in Stratagem would need to be rewritten, and I'm writing some network for another (commercial) project, network code which can be open-sourced

Does it do UDP and NIO?

['n ddrylliog]

For the Stratagem code : as far as I can remember, no. It uses TCP and plain old java Sockets

But that's precisely why I want to rewrite it all. For now I have looked into opportunities for two PCs to establish a direct connection without the need of a proxy server or special configuration of the router : it seems possible, with a few hacks. It has been done long ago for UDP, and for TCP it's still experimental and needs a "bonjour" server (I didn't get the whole thing clear).

Well, it seems we'd better use a "reliable" protocol upon UDP (using NIO of course), and buy on that basis. Does anyone know a good Java library for that ?

[Marvin Fröhlich]

Does anyone know a good Java library for that ?

What about the ones, Mathias linked above?

Marvin

[Pandaemonium]

Darkstar seems pretty good. Though, there are a couple questions I have about it that have yet to be answered (the guys at Darkstar have never responded to my email) before I decide to commit to it 100%.

My concern about Darkstar is the persistence layer. I already have my persistence layer coded for my game (using Postgres). I opted for a relational db so I can easily perform queries and develop an admin web tool to monitor the game. To me, having the ability to query the state of my game in real time is very important. Afaik, Darkstar achieves all of its persistence with the use of managed objects implementing Serializable (if there was a way to plop my own persistence layer in then I'd definitely like to know!)  .

So it should NOT serialize whole objects and transmit them between server and client

Forgive my possible ignorance on the subject, but why is it bad to serialize an object between client and server? I know there is some overhead involved with serializing, but what if care was taken to ensure that the size of such objects was very small? As an example I have a Request object that looks like this:

Code:

pseudo:
 
class Request
  enum Action
  enum Type
  Object value
end class Request

For the above request object, the server can (depending on the action) send an object to the client over the network. So far, this has been very smooth and fast. Is there something I'm forgetting to consider?

[Marvin Fröhlich]

So it should NOT serialize whole objects and transmit them between server and client

Forgive my possible ignorance on the subject, but why is it bad to serialize an object between client and server? I know there is some overhead involved with serializing, but what if care was taken to ensure that the size of such objects was very small? As an example I have a Request object that looks like this:

Code:

pseudo:
 
class Request
  enum Action
  enum Type
  Object value
end class Request

For the above request object, the server can (depending on the action) send an object to the client over the network. So far, this has been very smooth and fast. Is there something I'm forgetting to consider?

I haven't got too much experience with serializing. But doesn't Java create new instances any time it deserializes an object? This way we would get a huge GC problem.

Marvin

[Pandaemonium]

I haven't got too much experience with serializing. But doesn't Java create new instances any time it deserializes an object? This way we would get a huge GC problem.

Yes, a new instance is created upon deserialization. However, if done correctly, there should be no issues with GC. Here is an excerpt taken from smartdataprocessing.com, lesson 11:

-----
Serializaion in the Real World

In some types of applications you have to write the code to serialize objects, but in many cases serialization is performed behind the scenes by various server-side containers. These are some of the typical uses of serialization:

    * To persist data for future use.
    * To send data to a remote computer using such client/server Java technologies as RMI or socket programming.
    * To "flatten" an object into array of bytes in memory.
    * To exchange data between applets and servlets.
    * To store user session in Web applications.
    * To activate/passivate enterprise java beans.
    * To send objects between the servers in a cluster.

When you use serialization in time-critical applications, for example real- time stock trading systems, the size of the serialized objects should be minimal. Keep in mind that variables with longer names produce larger footprints during serialization, and this may substantially slow down your application. Think of a high volume of trade orders that is being serialized. I remember working on the application where a class TradeOrder had about a hundred member variables. After renaming the variables into meaningless v1, v2, and so on, the size of one TradeOrder instance was reduced by a thousand bytes. And we are talking about serializing of thousands orders over the network!

If performance is your primary goal, use Externalizable interface instead of Serializable. Yes, you'll have to write code to serialize each attribute, but this may speed up serialization process substantially.
-----


With my Request object, they can come in from the GameServer and are processed in a single method. Thus, if I did happen to have an object come in over the network and didn't do anything with it, then that object will be available for GC. Using the advice from above, I've taken care to minimize my Request object (and extra special care for those objects I place into the 'value' variable). 

Is there a case you can think of where GC will be an issue?

[Marvin Fröhlich]

Is there a case you can think of where GC will be an issue?

Yes. With GC-problems I didn't mean, that objects are not destroyed, but that a high amount of objects is destroyed by the GC. In ral-time apps like games, this is a problem. We encountered this problem with Sun's crappy vecmath, which is why we moved to vecmath-kh a year ago. Reducing the amount of GCed objects gets you rid of all the small hick-ups, your game will have otherwise.

So, in this sense Serializing is bad.

Marvin

[Pandaemonium]

Yes. With GC-problems I didn't mean, that objects are not destroyed, but that a high amount of objects is destroyed by the GC. In ral-time apps like games, this is a problem. We encountered this problem with Sun's crappy vecmath, which is why we moved to vecmath-kh a year ago. Reducing the amount of GCed objects gets you rid of all the small hick-ups, your game will have otherwise.

So, in this sense Serializing is bad.

Ahh, I see. I can see where you're coming from and I agree. In this context, it totally makes sense that you mentioned we would only want to transmit primitives (or a set thereof). With my Request object (continuing he example) I usually only populate the 'value' with primitive wrapper objects. I'll definitely need to consider the GC issue.

[kukanani]

Hey, meanwhile, while this is being worked on, what would be the best way to send a Transform3D over a Socket?  Should I parse it?

[Marvin Fröhlich]

Hey, meanwhile, while this is being worked on, what would be the best way to send a Transform3D over a Socket?  Should I parse it?

A Transform3D is fully defined by a Matrix4f(). So you would only need to send that matrix over the net. And a Matrix4f is of course fully defined by 16 floats. So you simply need to send 16 floats over the Socket.

Marvin

[kukanani]

Thanks.

['n ddrylliog]

I just wanted to leave a short note on my view of how it should be done.

In my opinion, this network API should use bindings (very much like JiBX for XML, for example), which says which values, and in which order, and which serializer is used (there is more than one way to serialize a map, for example) and then code is dynamically generated (very much like in JiBX).

In that case we have both flexibility and speed, along with ease of use. Do you see what I mean and would you go in this direction ?