MQ - Message Queueing Services P2P and PubSub


Thành viên VIP
(continue from HERE)

After finishing the boring chore I start with the generic topics for the introduction into Messaging Queueing Services. As you have already known that such services are a part of JAVA. The services are bundled in the package Java Message Service.


Now more than ever, JMS requires a CONTAINER PROVIDER that acts as the "CONTAINER". For example, BEA WebLogics which was originally founded by 3 men (B for Bill Coleman, E for Ed Scott and A for Alfred Chuang) is one of JMS Container Provider and was acquired by ORACLE (and JAVA of SUN). In general there are two types of Messaging Services:

- PEER-TO-PEER or POINT-TO-POINT: the communication service is between two "equal" participants (hence PEER). It's like the telephoning message between caller and callee. The buzzword for this service is the P2P. Whatever definition you like it is irrelevant.


- ONE-TO-MANY or PUBLISHER-SUBSCREIBERS: this type of services is, as its name says, a broadcasting of massages to the (unknown) subscribers. The buzzword for that is the PubSub.

(Source: Oracle/WebLogics)

So, the essence of this Voodoo technology is either to bring the messages from man to man (or app to app) or to hang the messages on the lamp post for those who concern so that "they" can pick up the "right" messages on their own. With this understanding it's quite easy to implement your own Messaging Services.

P2P with Socket-to-Socket. An Example: the communication between an App and a Database Server is the P2P communication. To implement P2P it is usually so:
    soc = InetSocketAddress(dbHost, port)); // or Socket
    soc.socket().setReceiveBufferSize(32768); // 32KB
and the "wrapped API" is roughly so:
ODBConnect odbc = new ODBConnect(host, port, password, uid);
This way is implemented similarly by all other DB servers, too.

PubSub with MulticastSocket-to-MulticastSocket. An example: the "so-called" mailing letters. One letters to all who "subscribe" (or register). To implement a PubSub it is relatively quite simple:
// Publisher:
    MulticastSocket mcs = new MulticastSocket(port);
    InetAddress group = InetAddress.getByName(host);
    while (loop) {
      while (msgLst.size() > 0) {
        byte[] msg = msgLst.remove(0);
        mcs.send(new DatagramPacket(msg, msg.length, group, port));
// Subscriber:
    MulticastSocket listener = new MulticastSocket(port); // Port of Publisher
    listener.joinGroup(InetAddress.getByName(host));       // JOIN the Publisher Group
    while (loop) { // max. paket 8 KB
      DatagramPacket packet = new DatagramPacket(new byte[8192], 8192);
      listener.receive(packet); // wait for the incoming msg
      byte[] bb = new byte[packet.getLength()]; // gauge the message size
      System.arraycopy(packet.getData(), 0, bb, 0, bb.length); // get it
      ... // do something with the message
As you see, the implementation for two Voodoo Magics P2P and PubSub requires only a few code lines. Why the fuss with IBM MQ or Rabbit MQ? Oh, you may ask for the Persistency. Am I right? Well, Persistency is as nebulous as the buzzword AI. It's lovely to observe the laymen who try to excel themselves (like some IT journalists) in the IT scene. And If they don't know how the Voodoo thing works they simply cover it with AI... as if AI is the most omnipotent solution for everything. More intelligent than themselves. But Persistence isn't as complicated as it sounds. Persistency is no other thing than to save the stuff in a file and to read it back when it is needed. Whatever the stuff is the main thing is that it has to be retrieved exactly as it was before. Well, a file or an expensive Database? It's your choice.

With a flat file it could be a hard work for a mediocre developer to reassemble the stuff back to its original form. What about a database? Object Oriented Database (OODB) is not so common and somehow too generic. Relational database (RDB) is like using a Saturn Rocket to kill a mosquito. Why? RDB requires an object to be "atomized" to a heap of unrelated components and the reassembling is as hard as the work with a flat file. However, every OO Programming Language (OOPL) gives the developers the most useful tool to save an object and to retrieve it exactly as it was and, of course, without a ton of sweat. It's the SERIALIZATION process. JAVA, C# and even PYTHON offer you this wonderful technique. Why don't use it? It's not only cheap, but also sweat-free.

So, the nebulous Persistence can be solved with a Serialization process. But real life is never a free lunch scenario. Otherwise all real IT gurus could be out of work. Serialized object distinguishes from other serialized objects and, to the worst, from OOPL to OOPL. Meaning that every object needs a serialized file and can be only reassembled by its creator OOPL. Several serialized files are probably not a real problem to handle and easier than to toil with hundreds or thousands of files. More serialized files = more troubles. You run really into a maze and ensnare yourself in the myriad serialized files.

Again, to solve the access problem with myriad serialized files it is better to save "them" in ONE file and then index each of them with an unique key. This way enables you to deal with ONE file instead of navigating among the mountains of files. And, the best of it is the fast access to the objects thanks to the file caching mechanism of the underlaying OS. Further, you can reduce the access time with buffering (e.g. in JAVA: BufferedInputStream). Example:

The saving process: Suppose that all the serialized objects are "indexed" by keys and stored in a HashMap map. The coding work could be as following:
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(fileName, true));
ByteArrayOutputStream bao = new ByteArrayOutputStream( );
List<String> list = new ArrayList<>(map.keySet());
for (String key : kLst) {
  // object Key: String, max. 32 characters (max. length 32)
  // object: serialized object (max. 4GB bytes big)
  ObjectOutputStream oos = new ObjectOutputStream(bao);
  byte[] objLen = new byte[6];
  // key. Conversion int to 2 bytes
  objLen[0] = (byte)(key.length() & 0xFF00 >> 8);
  objLen[1] = (byte)(key.length() & 0xFF);
  // serialized object from "map"
  // conversion to 4 bytes
  int len = bao.size();
  objLen[2] = (byte)(len & 0xFF000000 >> 24);
  objLen[3] = (byte)(len & 0xFF0000 >> 16);
  objLen[4] = (byte)(len & 0xFF00 >> 8);
  objLen[5] = (byte)(len & 0xFF);
  // the indexed object with a key
The retrieving process:
byte[] buf = Files.readAllBytes((new File(dbName)).toPath());
HashMap<String, Object> map = new HashMap<>();
for (int i = 0; i < buf.length; ) {
  // conversion 2 bytes to int
  int keyLen = (int)(buf[i] << 8 + buf[i+1]);
  // conversion 4 bytes back to int
  int objLen = (int)(buf[i+2] << 24 + buf[i+3] << 16 + buf[i+4] << 8 + buf[i+5]);
  // access the key and object
  String key = new String(buf, i+6, keyLen);
  ByteArrayInputStream bai = new ByteArrayInputStream(buf, i+6+keylen, objLen);
  ObjectInputStream ois = new ObjectInputStream(bai);
  // make it accessible via index (here: key)
  map.put(key, ois.readObject());
  // next element
  i += (6+keyLen+objLen);
With the HashMap map you can now access every object of any type. What you have to do is to cast the object to its proper form. Example with two distinctive objects: Employee and Invoice
public class Employee implements {
  private static final long serialVersionUID = 5678L;
  @param id     String, employee unique ID
  @param name   String, employee name
  @param birth  String, birthday dd/MM/yyyy
  @param gender String (Male/Female)
  @param address String
  @param mobile String, mobile phone
  @param email  String, email address
  @param category int, 0: Sale, 1: Purchasing, 2: Billing 3: HumanResource, 4:Product (superuser)
  @param picture String, image local file or URL (WEB) link
  public Employee(String id, String name, String birth, String gender, String address,
                       double salary, String mobile, String email, String category, String picture) { = id; = name;
    this.birth = birth;
    this.gender = gender;
    this.address = address;
    this.salary = salary; = mobile; = email;
    this.picture = picture;
    this.category = category;
  ...// getters, setters and some methods
public class Invoice implements {
  private static final long serialVersionUID = 9012L;
  @param date, String
  public Invoice(String date) { = date;
  ...// getters, setters and some methods
The access:
Employee employee_joe = (Employee)map.get("joe");
Invoice july_invoice = (Invoice)map.get("Tuition_of_July");
Even on a mediocre, cheap laptop the saving and retrieving process are much faster than saving or reading the objects to/from different files or to/from a RDB.

The Voodoo magics MQ is denuded and debunked. The spell is broken. Even the seem-to-be unsolvable, fully obfuscated Persistency becomes transparent and a child's play.

The last question you could raise is about the Database. An Object Oriented Database (OODB) to be exact. Well, The saving file is for the unknowledge a simple flat file, but for you, who created it, it is a file indexed by keys whereas the objects are the data. If you put this file under surveillance of an app called Server that enables other apps to access the data via keys with Socket communication (P2P) then you get a REAL Object Oriented Database. And if you enhance the accesses with some typical Database features like lock, unlock, commit, etc. your OODB is now a full-fledged OODB. With a simple or full-fledged OODB your P2P/PubSub Messages could be made "Persistence".


Up to this point one can see the reason why a real development work of a System Developer (S-Dev) is absolutely different from a MQ/RDB Application Developer (A-Dev). It's like the day from the night. Lots of companies complain wrongly about "expensive" IT A-Dev who thrive on some buzzwords like MQ, Rabbit, Cloud Foundry, ReactJS and so on and confuse them with the real S-Dev who are more creative and more innovative in the IT development process.

(continue HERE)
Sửa lần cuối: