HƯỚNG DẪN Object Database of Serialized Objects for Cloud Computing ?

Joe

Thành viên VIP
21/1/13
2,927
1,312
113
Hi

Time to revise the Thread "OODB Design And Implementation". Again, if you are a founder or co-founder of a Startup you need only to inbox me with your email address and I'll send you the sources.


Object-Oriented
Database
OODB
&
Cloud Computing


I. FOREWORD

Object Oriented Programming Languages (OOPL) are nowadays the preferred programming languages in Software Application Development. C++, C#, PYTHON or JAVA are usually the first OOPL choice. The OOPL Application data are accordingly Objects which could be plain (text) or complex (e. g. image, sound). However the main problem of OOPL Objects is that how these objects should be stored and how they could be retrieved exactly as they originally were. Conventional Relational Database (RDB) such as MySQL or Oracle requires the objects to be so atomized that they could only retrieved in SQL (Structured Query Language). That is the problem. Atomized objects are very inappropriate for object implementation. Because OOPL objects are OOPL dependent they are usually incompatible to each other so that an C# object is unusable in JAVA or vice versa. Therefore it is almost impossible to design and to build an Object-Oriented Database that should be generic for any kind of OOPL Objects.


II. OBJECT-ORIENTED DATABASE (OODB)

An attempt to build a generic OODB in an OOPL requires a physical segregation of the storing and the retrieving mechanism. The most common classic method is the Client-Server model wherein the server can be implemented in any OOPL (here JAVA) and the data are stored as they were without any modification. And that allows the clients to be developed in whatever OOPL using its own storing mechanism (serialization) and data model for the object. The storing data format is the Serialization of object. Serialized objects can be saved on disk so that they can be retrieved later and at any time. And they are the basic form for an OODB implementation.


III. OODB ARCHITECTURE

1611049864666.png

As mentioned, serialized object is OOPL specific and different from each other the data have to be bundled in (independent) bytes before they can be saved or retrieved (i. e. written to or read from a file).

Instead of writing to or reading from a file an implemented OOPL Client API sends or receives the serialized data byte-wise to or from a socket which is connected to an OODB Server over a network (e. g. the Internet.) Such a communication method segregates physically Clients from Server and that allows the Clients to be implemented in any OOPL by using its according API: Client serializes an Object, saves the serialized data using the Client APIs and retrieves them back via the APIs (as if it wrote to a flat file and read from it) whereas the Client APIs do the Send-Receive communication between Clients and OODB Server. Some examples in C#, JAVA and PYTHON

C# (C SHARP)

The codes
Code:
using System;
using System.IO;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;
using System.Threading.Tasks;
namespace C_SharpExample
{
  [Serializable]
  class Tutorial
  {
  public int ID;
  public String Name;
   static void Main(string[] args)
   {
    // create and initialize
    Tutorial obj = new Tutorial();
    obj.ID = 1;
    obj.Name = ".Net";
    // save or write to D:\Example.txt
    IFormatter formatter = new BinaryFormatter();
    Stream stream = new FileStream(@"D:\Example.txt",FileMode.Create,FileAccess.Write);

    formatter.Serialize(stream, obj);
    stream.Close();
    // retrieve or read from D:\Example.txt
    stream = new FileStream(@"D:\Example.txt",FileMode.Open,FileAccess.Read);
    Tutorial objnew = (Tutorial)formatter.Deserialize(stream);

    Console.WriteLine(objnew.ID);
    Console.WriteLine(objnew.Name);

    Console.ReadKey();
  }
}
}
Explanation:

C# Object (Source: https://www.guru99.com)
1611049975234.png

Serializing and Saving process
1611050020465.png

Reading (or deserializing) process
1611050093959.png

JAVA
The Code
Java:
// the object
public class Tutorial  implements java.io.Serializable {
  public int id;
  public String name;
}
// the application
import java.io.*;

public class JavaExample {
  public static void main(String[] args) throws Exception {
      // create and initialize
      Tutorial obj = new Tutorial();
      obj.id = 1;
      obj.name = "java";
      // save or write to C:\example.txt
      ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("Example.txt", false));
      oos.writeObject(obj);
      oos.close();
      // retrieve or read from C:\example.txt
      ObjectInputStream ois = new ObjectInputStream(new FileInputStream("Example.txt"));
      Tutorial tut = (Tutorial) ois.readObject();
      ois.close();
  
      System.out.println("ID:"+tut.id+" Name:"+tut.name);
  }
}
Explanation
1611050144607.png

PYTHON
The codes (PYTHON 3)
Python:
import pickle
tutorial={"id:":1, "name:":"Python"}

f=open("examplePython.txt","wb")
pickle.dump(tutorial,f)
f.close()

f=open("examplePython.txt","rb")
d=pickle.load(f)
print (d)
f.close()
Explanation
1611050189146.png

The Serialization algorithm shows us how an OOPL object is created, initialized and saved and how different it is from OOPL to OOPL. The following contents of PYTHON and JAVA serialized object underlines the difference between “serialized” data of PYTHON and JAVA:
1611050217865.png

As we could see, OOPL serialized data vary from each other to each other so that it is difficult to convert the data from one OOPL to another OOPL. Especially if serialized objects contain some executable methods (the reason is that methods are NOT included in the serialization process). Neither in C# nor in JAVA. To deserialize serialized objects with executable methods OOPL always requires an Object code frame (i. e. class) to cast a serialized object into an executable instantiated object. Therefore all the classes of serialized objects must always be bundled with the Client application.

Examples
C# object with some internal methods
1611050247702.png

1611050309830.png

JAVA Object with some methods
1611050345182.png
 
Sửa lần cuối:
  • Like
Reactions: ngtheanh.dev

Joe

Thành viên VIP
21/1/13
2,927
1,312
113
1611050484198.png

IV. OODB CLIENT-SERVER ARCHITECTURE

From the Client application point of view Object data are OOPL dependent. The conversion of serialized data between different OOPL won't make sense because the process is not only difficult, but also very time-consuming. It is better to keep the serialized data as they are and save them independently in an OODB. The client application can then reassemble (casting) the serialized data from the OODB to the original and executable OOPL object. In this way an OODB which can be implemented in any OOPL can stow, retrieve the data (in binary format) serialized by whatever OOPL the Clients use.

To do that some common rules have to be conventionalized:
  • The communication between OOPL Clients and DB Server must be strictly via Socket. The physical segregation allows applications to be implemented by any OOPL favored by clients.
  • A set of Network Commands and protocol (NCP) must be mutually agreed, followed and implemented.
  • Each OOPL possesses its own set of APIs which is implemented upon the agreed conventions: Socket and NCP.
  • Clients can be anywhere on the net: LAN, WAN and Internet (Cloud users).
  • Server is like Client and can be anywhere on the net: LAN, WAN or Internet. Clustered servers are possible.
  • Failsafe for the case of the client primary (i. e. main) server is down. The Client must be able to switch to the next node or simply to quit.
To implement the two last requirements Multicasting IP (broadcasting) or an IP pair (for each node -one is the one for the broadcaster, the other is for the listener.) The network protocol is the UDP (User Datagram Protocol).

Multicasting IP is usually used in Message Service (MS) as Publisher-Subscribers (PS). The other is then the Peer-to-Peer (PP) Message Service. For security reasons the messages (i. e. UDP data) should be encrypted before they are sent. The encrypting Algorithm could be your own development or any available public encryption algorithm.

On the Server site the Server could be implemented in any OOPL. The physical segregation between Clients and server allows an OOPL independent implementation. However the OOPL dependent data must be known and accessed ONLY in that OOPL by the clients because it is for the server the data are just an array of binary bytes.

The Server Network allows clients to access directly to any server or simultaneously to several servers on the net. An implemented OODB Event enables each server on the network to identify the status of other servers on the net whether what servers are running and what servers are/were down. And each Server must be able to access other servers on behalf of the client in case that the OODB is distributed on the network (i. e. on different servers in the network).

CAUTION
To access a local OODB Server from the outside the Router of the Server has to be configured to forward the Server Port. Any client outside the network (but not inside the network) can access the site using its WEB name or IP address (e. g. using whatismyipaddress.com to determine the WEB IP of the OODB Server). To reduce the traffic load on the net OODB server compresses the data before the data are sent when, and only when, the data size is larger than 256 bytes.
A Client API Package in other OOPL than JAVA needs to implement the receiving Byte Stream (as byte array) with the following Byte-Stream layout into the according OOPL (e. g. PYTHON, C#, etc.):

1611147886734.png

The OODB Client-Server Architecture: any client can simultaneously access to any server (or several or all servers).

1611147937773.png

If a client application is implemented with the features OBDEvent, ODBEventListening and ODBEventListener it could be automatically triggered by ODBEvent for an intervention when one of the connected OODB Servers is, for example, down. In such case the Client is relatively failsafe (more: see Chapter VI. Object-Oriented DB Server Implementation.)

Example in JAVA-JFX:
Java:
public class eDict extends Application implements ODBEventListening {
  public void start(Stage stage) {
     ...
     try {
      int port = Integer.parseInt(pl.get(3));
      dict = new eOpenDict(pl.get(0),pl.get(1), pl.get(2), port);
      dict.register(this); // register to ODBEventListener
    ...
  }
  ...
  public void odbEvent(ODBEvent event) { // implemented the ODBEventListening
    Platform.runLater(() -> {
      String msg = event.getMessage();
       ...
       Alert alert = new Alert(AlertType.CONFIRMATION);
       alert.setTitle("SEVERE PROBLEM @"+host_port);
       alert.setHeaderText("FAILSAFE: Switch to the NEXT Server Node, OK?");
       alert.setContentText("Your Choice");         
       Optional<ButtonType> result = alert.showAndWait();
       if (result.get() != ButtonType.OK) exit();
       if (!dict.switchNode(event)) {                  // process the event
          alert = new Alert(AlertType.ERROR);
          alert.setTitle("ALERT");
          alert.setHeaderText("This is an EMERGENCY Message");
          alert.setContentText("Unable to start any backup Server!");         
          alert.showAndWait();
          exit();
        }
        dict.register(This);                        // re-register to the “fallen-in” new server
        host_port = dict.getActiveNode();
        txtArea.setText("Node "+node+" is down."+
                               "\nSwitch to Node "+host_port+
                               "\nKeyList AutoRefresh.\n");
      }
      ...
    });
  }
  …
Java:
public class eOpenDict {
  ...
  // fail-safe
  public boolean switchNode(ODBEvent e) {         // process the ODBEvent
    try {
      ...
      odbc = e.onEvent(pw, uid, dbList);                // execute the ODBEvent
      activeNode = e.getActiveNode();
      return (odbc != null);
    } catch (Exception ex) { }
    return false;
  }
  ...
  public void register(ODBEventListening odbe) {
    odbc.register(odbe);
  }
  ...
And the result is as following:

1611148041702.png

And when the principal Server was down. ODBEventListener triggers a POP-UP for an action

1611148080264.png
The Switch to the NEXT active Server.

Note that the DB upper Part on Server 9999 got lost and the keywords start with “many” instead of “abroad”. The “localhost” could be the WebHostName (e. g. congdongjava.com) or the WebIP (e.g. Congdongjava WebIP 104.199.133.223) if the ODB server is attached to the Web (see the aforementioned Caution).
 
Sửa lần cuối:
  • Like
Reactions: ngtheanh.dev

Joe

Thành viên VIP
21/1/13
2,927
1,312
113
1611148352080.png
In case that all Servers on the net were down this POP-UP shows up.

If the OODB is spread all over several servers the principal server where the Client is directly connected will automatically try to verify for the online status of all other connected servers and then acts as the Proxy Client to access the missing OODB parts on these servers (unaware by the client.) The OODB is then “distributed” among the network: Distributed OODB.

The above given example bases on the eDict Distributed OODB whose first part is on the localhost:9999 and the second part on the localhost:8888. If both servers are attached to the Internet their name could be any WebName (instead of localhost) or any IP.

NON-JAVA OOPL client can only save and retrieve serialized objects using the implemented OOPL Serialization Algorithm. The receiving data stream contains various fields which are determined by the given lengths in the following sequence:
  • 1 byte (first byte) represents the boolean values: TRUE is 0x01 and FALSE 0x00. Default 0x00
  • up pos. 1: 4 next bytes contain an Integer value. All 0x00 if an Integer is absent
  • up pos. 5: 2 next bytes contain the length of a replied message. Both are 0x00 if no message is available (a NULL string)
  • up pos. 7: 2 next bytes contain the length of an error message. Both are 0x00 if no error occurs on the server site (a NULL string)
  • up pos. 9: 4 next bytes contain the total length of a (String) List (note: not the number or size of items). All 0x00 if NO list is needed (a NULL list)
  • up pos. 13: 4 next bytes contain the length of a serialized object in byte array. All 0x00 if no object is a NULL
  • up pos. 17: n next bytes represent the message. If message is null n = 0
  • up pos. 17+n: m next bytes represent the error. If error is null m = 0
  • up pos. 17+n+m: l next bytes represent the String-List. l = 0 if String-List is null
  • up pos. 17+n+m+l : x next bytes represent the serialized object
Up the position 17 it could be either a message, or an error message (n = 0), or the String-List (n = m = 0) or the serialized object (n = m = l = 0). The Stream size: min. 17 bytes, max. unlimited. An example in C# with byte array which contains the serialized C# object:
Code:
// convert a C# serialized object into a byte array
try {
  byte[] byteArray;
  var formatter = new BinaryFormatter();
  using (var ms = new MemoryStream()) {
    using (var ds = new DeflateStream(ms, CompressionMode.Compress, true)) {
      formatter.Serialize(ds, set);
    }
    ms.Position = 0;
    byteArray = ms.GetBuffer();
  }
} catch (Exception ex) {
  ...
}
Code:
// deserialize a C# serialized object from a byte array
var set = new DataSet();
try {
  var formatter = new BinaryFormatter();
  using (var ms = new MemoryStream(byteArray)) {
   using (var ds = new DeflateStream(ms, CompressionMode.Decompress, true)) {
     set = (DataSet)formatter.Deserialize(ds);                     
   }
  }
} catch (Exception ex) {
     ...
}
The receiving byte array object need only to be deserialized by the Client in C#.

As mentioned, the receiving data can be compressed and needs to be inflated before the data can be used. The GZIP algorithm is applied to compress the data. In order to recognize the receiving data whether they are compressed in GZIP or not the first ten bytes in hex 1F8B0800000000000000 indicate the GZIP compressed content. Example:
Code:
0        4        8        A        E
1F8B0800 00000000 00008D90 3B0E0241 0C43FB3D | .... .... .... ;..A .C.= | 0
4DECFCEF 7F31BC03 121420D1 8C9CE4C9 8927898E | M... .1.. .. . .... .'.. | 1
318CEF4B 7539919E 44658DCD A0101D81 A4D9E921 | 1..K u9.. De.. .... ...! | 2
B7A45AF5 36C7BACF E872D05D 4FB90783 998C4693 | ..Z. 6... .r.] O... ..F. | 3
610DF71D 464C312B 9E9C5C27 801E98F1 1AB7F55E | a... FL1+ ..\' .... ...^ | 4
3902368D 5437DCD4 D696E27D 88C0A362 106FF837 | 9.6. T7.. ...} ...b .o.7 | 5
97BAC542 BB34BA26 6DA7CC26 6FA5933F CB5A99AC | ...B .4.& m..& o..? .Z.. | 6
B86F4846 68CAB463 23CBDDC2 280D3AB3 B10B3B25 | .oHF h..c #... (.:. ..;% | 7
8D0A0B4F ED0FA5FD 83438E92 D0717FDF 034BFFD6 | ...O .... .C.. .q.. .K.. | 8
748B0100 00                                  | t... .                   | 9
 
Sửa lần cuối:
  • Like
Reactions: ngtheanh.dev

Joe

Thành viên VIP
21/1/13
2,927
1,312
113
V. CLIENT API IMPLEMENTATION

JAVA is still the most portable OOPL of today. The OODB implementation in JAVA is split into two parts:

1. Client APIs (should be ported into other OOPL if needed)
2. Server APIs (should not be ported)

To port and to implement the Client APIs we need strictly to follow the agreed conventions. A NCP Byte Stream of x bytes which is sent over the network to an OODB Server.
1611404949823.png

The receiving byte stream on the Client Site is as following:
1611405024005.png

Layout of List/Array elements: len (2 bytes) element (len bytes) ….
1611405095350.png

In order to connect successfully to an OODB Server which is implemented in JAVA five Client APIs need to be implemented:

1. SocketConnection and Communication API. JAVA Implementation: ODBConnect
2. EventListening Interface API. JAVA Implementation: ODBListening. For C# (click HERE) or PYTHON (click HERE).
3. EventListener API: A Network Implementation Multicast Socket UDP listening thread. JAVA Implementation: ODBEventListener
4. Event API: an Event Implementation. JAVA Implementation: ODBEvent.
5. EnCrypting/Decrypting API: see the JAVA implementation


SocketConnection & Communication

ODBConnect is the Socket(-Channel) implemented communication API. ODBConnect verifies the Read-Write-Delete Privileg on the client site and that releaves the processing load on the server site. It's the part of Load-Balancing between Clients and Server.

Constructor.
Synopsis:
Connect(Connect(String dbHost, int port, String pw, String uID) throws Exception
Network Command Stream:
1 byte for CMD = 0x00
2 bytes for the length of encrypted String PW:ID
n bytes for the encrypted String PW:ID
Expected Data Receiving Stream:
1 byte contains 0x01 for Successful or 0x00 for failure
4 bytes contain 0.
2 bytes contain the length n of replied Message
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain the length x of the list of cluster nodes
4 bytes contain 0.
n bytes contain the Message (see image below)
e bytes contains the Exception message
x bytes contain the nodes of the list. Each key starts with 2 bytes node Length nLen
and followed by nLen bytes.

1611405151321.png
The first byte of the Data Receiving Stream is the UserPrivilege (0: Read only, 1: Read-Write,
2: Read-Write-Delete), the next n bytes for the assigned UserID (ID+@+7 hex. Digits) and the rest for the MulticastIP : Port (see image, eventually used by EventListener). To balance the load between Client and Server the User Privilege is used by SocketConnection (JAVA package: ODBConnect) to keep track of the processing privilege of a request while the server just processes it.

Connect an dbName to ODBServer
Synopsis:
void connect(String dbName) throws Exception
Network Command Stream:
1 byte for CMD = 17 (or 0x11)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
n bytes contain the Exception Message

Disconnect from ODBServer
Synopsis:
void disconnect( )
Network Command Stream
1 byte for CMD = 2 (or 0x02)
2 bytes conatain the length 1 (or 0x0001)
1 byte contains an asterisk *
Expected Data Receiving Stream: ignored

GetKeys retrieve all accessing keys of objects (incl. keys, local and from other nodes)
Synopsis:
List<String> getKeys(String dbName)
Network Command Stream:
1 byte for CMD = 3 (or 0x03)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain length x of the key list
4 bytes contain 0.
e bytes contain the Exception Message if there is one
x bytes contain the keys of the list. Each key starts with 2 bytes Key Length kLen and
followed by kLen bytes.


GetLocalKeys retrieve all accessing keys of objects on the connected local ODBServer
Synopsis:
List<String> getLocalKeys(String dbName)
Network Command Stream:
1 byte for CMD = 4 (or 0x04)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain length x of the key list
4 bytes contain 0.
e bytes contain the Exception Message if there is one
x bytes contain the keys of the list. Each key starts with 2 bytes Key Length kLen and
followed by kLen bytes.

GetClusterKeys retrieve all accessing keys of objects on the given Node ODBServer
Synopsis:
List<String> getLocalKeys(String dbName, String node)
Network Command Stream:
1 byte for CMD = 5 (or 0x05)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the node (WebHostName : Port)
n bytes for the node

Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain length x of the key list
4 bytes contain 0.
e bytes contain the Exception Message if there is one
x bytes contain the keys of the list. Each key starts with 2 bytes Key Length kLen and
followed by kLen bytes.

Add (serialized) Object to dbName
Synopsis:
void add(String dbName, String key, Object obj) throws Exception
Network Command Stream:
1 byte for CMD = 6 (or 0x06)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length k of the key
k bytes for the node
y bytes for the object which is converted to y bytes (or byte array)
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Unlock (serialized) all Keys of dbName
Synopsis:
boolean unlock(String dbName)
Network Command Stream:
1 byte for CMD = 7 (or 0x07)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0x01 for TRUE and 0x00 for FALSE
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Update (serialized) Object of dbName
Synopsis:
void update(String dbName, String key, Object obj) throws Exception
Network Command Stream:
1 byte for CMD = 8 (or 0x08)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length k of the key
k bytes for the node
y bytes for of the new object which is converted to y bytes (or byte array)
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Delete an object with the given key from dbName
Synopsis:
void delete(String dbName, String key) throws Exception
Network Command Stream:
1 byte for CMD = 9 (or 0x09)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the key
n bytes for the key

Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Read an object with the given key from dbName
Synopsis:
byte[] read(String dbName, String key) throws Exception
Network Command Stream:
1 byte for CMD = 10 (or 0x0A)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the key
n bytes for the key

Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain the length of the object byte array (which can be cast back to Object)
e bytes contain the Exception Message if there is one
n bytes of object byte array
IsExisted queries the existence of an object with the given key from dbName
Synopsis:
boolean isExisted(String dbName, String key)
Network Command Stream:
1 byte for CMD = 11 (or 0x0B)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the key
n bytes for the key

Expected Data Receiving Stream:
1 byte contains 0x01 for TRUE and 0x00 for FALSE
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

IsLocked queries an object with the given key from dbName if it is locked
Synopsis:
boolean isLocked(String dbName, String key)
Network Command Stream:
1 byte for CMD = 12 (or 0x0C)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the key
n bytes for the key

Expected Data Receiving Stream:
1 byte contains 0x01 for TRUE and 0x00 for FALSE
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Lock locks an object with the given key from dbName
Synopsis:
boolean lock(String dbName, String key)
Network Command Stream:
1 byte for CMD = 13 (or 0x0D
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the key
n bytes for the key

Expected Data Receiving Stream:
1 byte contains 0x01 for Successful and 0x00 for Failed
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Unlock unlocks an object with the given key from dbName
Synopsis:
boolean unlock(String dbName, String key)
Network Command Stream:
1 byte for CMD = 14 (or 0x0E)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the key
n bytes for the key

Expected Data Receiving Stream:
1 byte contains 0x01 for Successful and 0x00 for Failed
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Rollback rollbacks an object with the given key from dbName. If key is missing all objects of dbName are rollbacked.
Synopsis:
boolean rollback(String dbName, String key)
Network Command Stream:
1 byte for CMD = 15 (or 0x0F)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length n of the key (or missing)
n bytes for the key (or missing)

Expected Data Receiving Stream:
1 byte contains 0x01 for Successful and 0x00 for Failed
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Close dbName
Synopsis:
void close(String dbName) throws Exception
Network Command Stream:
1 byte for CMD = 18 (or 0x12)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
n bytes contain the Exception Message

Save dbName by commit all modifications and free all locked keys
Synopsis:
void save(String dbName) throws Exception
Network Command Stream:
1 byte for CMD = 19 (or 0x13)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
n bytes contain the Exception Message

xDelete (serialized) Object of dbName by lock key, delete then unlock key
Synopsis:
void xDelete(String dbName, String key) throws Exception
Network Command Stream:
1 byte for CMD = 20 (or 0x14)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length k of the key
k bytes for the node
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

xUpdate (serialized) Object of dbName by lock key, update then unlock key
Synopsis:
void xUpdate(String dbName, String key, Object obj) throws Exception
Network Command Stream:
1 byte for CMD = 21(or 0x15)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length k of the key
k bytes for the node
y bytes for the object which is converted to y bytes (or byte array)
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Commit (serialized) Object of key of dbName
Synopsis:
boolean commit(String dbName, String key)
Network Command Stream:
1 byte for CMD = 22 (or 0x16)
2 bytes for the length x of dbName
x bytes for the dbName
2 bytes for the length k of the key
k bytes for the node
Expected Data Receiving Stream:
1 byte contains 0x01 for Successful and 0x00 for Failed
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

Commit all (serialized) Objects of dbName
Synopsis:
boolean commit(String dbName)
Network Command Stream:
1 byte for CMD = 22 (or 0x16)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0x01 for Successful and 0x00 for Failed
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

autoCommit set Auto-Commit for dbName in case of emergency
Synopsis:
void autoCommit(String dbName)
Network Command Stream:
1 byte for CMD = 22 (or 0x16)
2 bytes for the length x of dbName
x bytes for the dbName
Expected Data Receiving Stream:
1 byte contains 0
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one

ChangePassword allows Client to change the Password (encrypted)
Synopsis:
boolean changePassword(String oldPW, String newPW)
Network Command Stream:
1 byte for CMD = 28 (or 0x1C)
2 bytes for the length x of old Password
x bytes for the old Password
2 bytes for the length of new Password
y bytes for the new Password
Expected Data Receiving Stream:
1 byte contains 0x01 for Successful and 0x00 for Failed
4 bytes contain 0
2 bytes contain 0
2 bytes contain 0. If NOT 0 (length e) it is an Exception
4 bytes contain 0
4 bytes contain 0.
e bytes contain the Exception Message if there is one
 
Sửa lần cuối:
  • Like
Reactions: ngtheanh.dev

Joe

Thành viên VIP
21/1/13
2,927
1,312
113
ODBMining is an extension of ODBConnect and is to facilitate the accesses of ALL ODB objects in the SQL manner. Beside the above-mentioned methods following methods are added:
Java:
Constructor:
    ODBMining(String dbHost, int port, String pw, String uID) throws Exception

Methods:
    String getClassName(String dbName, String key) throws Exception
    ArrayList<String> allClassNames(String dbName) throws Exception
    ArrayList<Class<?>> allClasses(String dbName) throws Exception
    ArrayList<Object> allObjects(String dbName)
    ArrayList<Object> allObjects(String dbName, String clsName)
    static Object exec(Object o, String m, Class<?> types[], Object[] parms) throws Exception
    ArrayList<Object> SQL(String dbName, String SQL_String) throws Exception
    ArrayList<Object> selectAll(String dbName, String pat) throws Exception
    ArrayList<Object> selectAll(String dbName, String vName, String pat) throws Exception
    ArrayList<Object> selectAll(String dbName, String comp, double cVal) throws Exception
    ArrayList<Object> selectAll(String dbName, String comp, float cVal)  throws Exception
    ArrayList<Object> selectAll(String dbName, String comp, long cVal) throws Exception
    ArrayList<Object> selectAll(String dbName, String comp, int cVal) throws Exception
    ArrayList<Object> selectAll(String dbName, String comp, short cVal) throws Exception
    ArrayList<String> getFieldNames(String dbName, Class<?> cls) throws Exception
    ArrayList<String> getFieldNames(String dbName, Object obj) throws Exception
    ArrayList<String> getFieldNames(String dbName, String key) throws Exception
    Object getField(String dbName, Object obj, String vName) throws Exception
    Object getField(String dbName, String key, String vName) throws Exception
Note: the SQL_String of the method SQL() should be formulated similarly to a SQL-SELECT statement. It begins with “select” then the conditional queries in Lower or Upper cases (except dbName and variable names). Example: “select name eq J*e and age gt 50 and income gt 20000” is different to “select Name eq J*e and Age gt 50 and Income gt 20000”. Comparators are lt, le, eq, ge, gt and Conjunctors are: and, or.

- The pattern (pat) of all select-Methods can be embedded with ONE wildcard which is either an asterisk ( * ) for a string or a question mark (?) for a single letter. Example:
SQL:
selectAll(myDB, “J*e”);        // select all objects which start with J and end with e (key name or content of
                               // any string field variable
selectAll(myDB, name, “Joe*”); // select all objects which has the field variable name and this variable contains a
                               // string that begins with Joe
- Methods with the parameter comp (for comparator) which must be eq, lt, le, gt and ge (either lower or upper case: equals, less-than, less-or-equal, greater-than, greater-or-equal). The same for the comparator within the SQL string. Example:
SQL:
selectAll(myDB, income, GE, 10000); // select all objects with the variable income which is Greater-Than 10000
- Method SQL allows you to formulate a SQL-like string. Example:
SQL:
SQL(myDB, ” select name eq J* and age lt 50 or income gt 10000”); // select all objects which start with  J* (variable
                                                                  // name)  and age (variable age) lt 50 or income (variable income)
                                                                  // greater-than 1000000
people_2.png

EventListening is an Interface which must be implemented if events on the server site should be listened.

Synopsis (in JAVA syntax):
Java:
public interface EventListening {
  public void odbEvent(Event e);
}

OdbEvent is an object which is instantiated by EventListener thread and passes it over to the app which implements EventListening. The mandatory implementation is a Constructor and the method getMessage.

Synopsis (in JAVA syntax):
Java:
public class ODBEvent {
    public ODBEvent(String msg) {
       this.msg = msg;
    }
    public String getMessage() {
       return msg;
    }
    private String msg;
}
ODBEvent can be expanded if more methods are needed.

EventListener should be an independent thread and is started by the (main) application, then it runs along with the main app till it is terminated by the main app. The EventListener is an implementation of MulticastSocket. MulticastSocket is useful for sending and receiving IP-Multicast packets. A MulticastSocket is a DatagramSocket, with the additional capabilities for joining "groups" of other multicast hosts (on the internet). More about MulticastSocket: see MulticastSocket API and about Global Internet Multicast IP.

Synopsis (in JAVA syntax):
Java:
public class EventListener implements Runnable {
  public ODBEventListener(String host_port) {
    ...
  }
  public void addListener(ODBEventListening odbe) {
    ...
  }
  public void run() {
    ...
    try (MulticastSocket dbListener = new MulticastSocket(port)){
       dbListener.joinGroup(InetAddress.getByName(host));
       … // do the receiving, unzip and decrypt the message, instantiate ODBEvent and call back to registered Clients
    } catch (Exception ex) { }
  }
  ...
}
ENCRYPTING/DECRYPTING Algorithm

Because the client has to exchange ID and Password with the OODB Server it is inevitable that the ID and the Password must be encrypted. Further, broadcasting Messages should be encrypted on the server site too so that the Clients must be able to encrypt/decrypt the messages with the same algorithm used by the server. However, if the algorithm is public and open this algorithm is more or less useless. From this point of view the Clients should use the Encrypting/Decrypting algorithm used by the server. Therefore the Client Package always includes the binary codes of the Encrypting/Decrypting algorithm: The EnDecrypt API.
Java:
Synopsis:
    public static String encrypt(String input)
Parameters:
    input is the string to be encrypted
Return:
    Encrypted String

Synopsis:
    public static String decrypt(String input) throws Exception
Parameters:
    input is the encrypted string to be decrypted
Return:
    decrypted String (plain-readable String)
OOPL other than JAVA (e. g. C# or PYTHON) should use the available Open Sources to access the two mentioned JAVA methods. Some Open Sources for C#: C#Corner or JAVONET and for PYTHON: JPype of SourecForge or Jpy


Access Rights and Privilege

By nature Clients of OODB are usually different. Administrators have more right (or privilege) than the newbies or senior developers. The access rights are:

0 Read only.
1 Read-Write on local Server only. This privilege allows the users to create a new ODB
2 Read-Write-Delete on every server (local or remote).
3 Superuser with Privilege 2

Note: up Privilege 1 allow the user to create a new ODB only on the local server. The Entry-format of the userlist (gzip and encrypted)
Code:
Password:UserID@Privilege
An example: the full implementation of EventListener in JAVA is ODBEventListener
Java:
public class ODBEventListener implements Runnable {
  /**
  @param host_port string containing MultiCast IP and port. Example 224.0.0.3:9990
  */
  public ODBEventListener(String host_port) {
    this.host_port = host_port;
  }
  /**
  @param host_port string containing MultiCast IP and port. Example 224.0.0.3:9990
  @param nodeList List of nodes (WebHostIP:Port)
  */
  public ODBEventListener(String host_port, List<String> nodeList) {
    this.host_port = host_port;
    this.nodeList = nodeList;
  }
  /**
  @param odbe ODBEventListening, implemented object
  */
  public void addListener(ODBEventListening odbe) {
    if (!set.contains(odbe)) set.add(odbe);
  }
  /**
  @param odbe ODBEventListening, implemented object
  */
  public void removeListener(ODBEventListening odbe) {
    set.remove(odbe);
  }
  public void exitListening() {
    listened = false;
  }
  public void run() {
    new Thread() {            // internal Thread that does the Instantiation and Callback
      public void run() {
        ODBOutputStream oos = new ODBOutputStream();
        while (listened) {
          try {
            while (msgLst.size() > 0) {
              oos.reset();
              byte[] buf = msgLst.remove(0);
              GZIPInputStream gi = new GZIPInputStream(new ODBInputStream(buf));
              for (int le = gi.read(buf); le > 0; le = gi.read(buf)) oos.write(buf, 0, le);
              gi.close(); //marking EOF gzip
              // Instantiate ODBEvent and callback
              ODBEvent event = new ODBEvent(EnDecrypt.decrypt(new String(oos.toByteArray())), nodeList);
              for (ODBEventListening odbe : set) odbe.odbEvent(event);
            }
            TimeUnit.MILLISECONDS.sleep(1);
          } catch (Exception ex) { }
        }
      }
    }.start();
    int port = host_port.indexOf(":");
    String host = host_port.substring(0, port);
    port = Integer.parseInt(host_port.substring(port+1));
    try (MulticastSocket dbListener = new MulticastSocket(port)){
       dbListener.joinGroup(InetAddress.getByName(host));
       while (listened) {
         DatagramPacket packet = new DatagramPacket(new byte[256], 256);
         dbListener.receive(packet); // wait for the incoming msg
         msgLst.add(packet.getData());
       }
    } catch (Exception ex) { }
  }
  private String host_port;
  private List<String> nodeList;
  private volatile boolean listened = true;
  private volatile Set<ODBEventListening> set = new CopyOnWriteArraySet<ODBEventListening>( );
  private volatile List<byte[]> msgLst = Collections.synchronizedList(new ArrayList<byte[]>());
}
And an implementation of ODBEvent in JAVA
Java:
public class ODBEvent {
  /**
  Constructor
  @param msg String the EventMessage
  @param nodeList List of all nodes (WebHostIP:Port)
  */
  public ODBEvent(String msg, List<String> nodeList) { // the 2. parameter is here optional
    this.msg = msg;
    this.nodeList = nodeList;
    off = msg.indexOf(":") < msg.indexOf(" ") && msg.endsWith("OFFLINE") && nodeList != null;
  }
  /**
  getMessage
  @return String, the EventMessage
  */
  public String getMessage() {
    return msg;
  }
  /**
  getActiveNode
  @return String, the running node (Host:Port)
  */
  // optional to show the readers how to extend an ODBEvent Object
  public String getActiveNode() {
    return node;
  }
  /**
  onEvent: connect to the next online host. As a fail-safe OODB the next OODB Server
  in the queue
  @param pw Password
  @param uid UserID
  @param dbList ArrayList containing all OODB names
  @return ODBConnect object, null for no connection or no alternative host was found
  */
  // optional to show the readers how to extend an ODBEvent Object. Here to switch the node
  public ODBConnect onEvent(String pw, String uid, List<String> dbList) {
    if (off) { // OFFLINE
      node = msg.substring(0, msg.indexOf(" "));
      if (nodeList.contains(node)) nodeList.remove(node);
      for (int p = nodeList.size(), b = 0; b < p; ++b) try {
        node = nodeList.get(b);
        if (node == null) return null;
        int l = node.indexOf(":");
        ODBConnect odbc = new ODBConnect(node.substring(0, l), Integer.parseInt(node.substring(l+1)), pw, uid);
        for (String dbName : dbList) { // reconnect all DBs
          try { // connect to the next node
            odbc.connect(dbName);
          } catch (Exception e) { }
        }
        return odbc; // done
      } catch (Exception ex) { }
    }
    node = null;
    return null;
  }
  //
  private List<String> nodeList;
  private String msg, node;
  private boolean off;
}
To implement the Event and EventListening, EventListener in other OOPL than JAVA, PYTHON or C# you could consult the following tutorials:

PYTHON: Event-driven programming in Python (click HERE for more details)
C#: Events or Event Handling in .NET using C# (click HERE or HERE for more details)

For more about your specific OOPL you could google with the key “Event and Event Handling in C# (or PYTHON)”. The implementation of ODBEvent depends fully on your specific needs and the purpose depends also on the requirements how the event is processed. And the same is valid with the EventListener.

And an example showing how the APIs are applied with EventListening, EventListener and Event. The JavaFX frame:
Java:
public class eDict extends Application implements ODBEventListening {
  public void start(Stage stage) {
    Application.Parameters params = getParameters();
    java.util.List<String> list = params.getRaw();
    ...
    try {
      int port = Integer.parseInt(list.get(3));
      //                                  uID              Password      WebHostName  port
      dict = new eOpenDict(list.get(0),list.get(1), list.get(2), port);
      ...
    } catch (Exception ex) {
      ...
    }
    dict.register(this);
    …
  // implement the ODBEvent-----------------------------------------------------
  public void odbEvent(ODBEvent event) {
    Platform.runLater(() -> {
      String msg = event.getMessage();
      if (msg.endsWith("READY") || msg.endsWith("OFFLINE")) {
        String node = msg.substring(0, msg.indexOf(" "));
        if (msg.endsWith("OFFLINE")) {
          cluster.remove(node);
          if (host_port.equals(node)) {
            Alert alert = new Alert(AlertType.CONFIRMATION);
            ...
            Optional<ButtonType> result = alert.showAndWait();
            if (result.get() != ButtonType.OK) exit();
            if (!dict.switchNode(event)) {
               ...
            }
          ...
    });
}
...
The eOpenDict class
Java:
public class eOpenDict {
  public eOpenDict(String uid, String pw, String host, int port) throws Exception {
    this.pw = pw;
    this.uid = uid;
    odbc = new ODBConnect(host, port, pw, uid);
    odbc.connect(dict);
    odbc.autoCommit(dict);
    words = odbc.getSortedKeys(dict);
  }
  ...
  public void register(ODBEventListening odbe) {
    odbc.register(odbe);
  }
  ...
  // fail-safe
  public boolean switchNode(ODBEvent e) {
    try {
      ArrayList<String> dbList = odbc.getDBList();
      odbc.disconnect(); // kill the zombie......
      odbc = e.onEvent(pw, uid, dbList);
      activeNode = e.getActiveNode();
      return (odbc != null);
    } catch (Exception ex) { }
    return false;
}
And the result
failsafe_1.png

An Example with a complex Java Serializable Object. The Object is “People.java” with SWING Dialog Implementation to display Image accessed from the Web:
Java:
public class People implements java.io.Serializable {
  private static final long serialVersionUID = 1234L;
  public People(String name, int age, String address,
                String profession, double income, String url) {
    this.age = age;
    ...
  }
  ...
  public void print() {
    ...
  }
  public void picture(JFrame jf) {     // SWING JDialog
    class Picture extends JDialog {
      public Picture(JFrame jf) {
        super(jf, true);
        setTitle("People:"+name);
        setDefaultCloseOperation(DISPOSE_ON_CLOSE);
        JButton but;
        try {
          ImageIcon img;
          if (url.indexOf("://") > 0) img = new ImageIcon(new URL(url));
          else img = new ImageIcon(url);
           if (img != null) but = new JButton(img);
           else but = new JButton("No Image available");
           setSize(img.getIconWidth(), img.getIconHeight());
        } catch (Exception e) {
           but = new JButton("No Image");
           pack();
        }
        but.setContentAreaFilled(false);
        but.setBorderPainted(false);
        add(but, BorderLayout.CENTER);
        setLocationRelativeTo(jf);
      }
    }
    (new Picture(jf)).setVisible(true);
  }
}
people_1.png

Note: the DB file userlist in directory Nodes/oodb_Node1 of the delivered Client Package contains the default Superuser (ID: admin, PW: system. Both in lowercase). With this default you can start the ODBServer (SWING: JODBServer.jar, JFX: JfxODBServer.jar), then move to the UserMaintenance Tab and create for yourself your own ID and PW. In case of corrupted userlist you need only to delete the file userlist and start ODBServer then the server creates a new userlist with the default Superuser admin-system. The superuser "admin/system" is used only to create your own userlist and cannot be used to access any ODBServer on the net (rejected by ODBWorker).
 
Sửa lần cuối:
  • Like
Reactions: ngtheanh.dev

Joe

Thành viên VIP
21/1/13
2,927
1,312
113
The JAVA Client API package (clientODB.jar) includes the following APIs:

1. ODBConnect: the interface between Client App and Network (Source & binary)
2. ODBInputStream: similar to ByteArrayInputStream (Source & binary)
3. ODBOutputStream: similar to ByteArrayOutputStream (Source & binary)
4. ODBReceiveStream: an Implementation to facilitate the Receiving Data Stream (Source & binary)
5. ODBEvent: the Event Object (Source & binary)
6. ODBEventListener: the EventListener (Source & binary)
7. ODBEventListening: the Event Interface (Source & binary)
8. EnDecrypt: the Encrypting/Decrypting Algorithm (only binary)
9. ODBMining: an extension of ODBConnect to enhance some basic functions of different Select Proceduresimilar to some SQL-Selects (optional, Source & binary)

Examples with the Encrypt/Decrypt Algorithm.
Java:
import joeapp.odb.EnDecrypt;
public class TestEncrypt {
  public static void main(String... a) {
    String en = EnDecrypt.encrypt(a[0]);
    System.out.println("Given:"+a[0]+"\nEncrypted:"+en);
  }
}
Java:
import joeapp.odb.EnDecrypt;
public class TestDecrypt {
  public static void main(String... a) throws Exception {
    String de = EnDecrypt.decrypt(a[0]);
    System.out.println("Given:"+a[0]+"\nDecrypted:"+de);
  }
}
EnDecrypt.png
Java:
import joeapp.odb.EnDecrypt;
public class TestEnDecrypt {
  public static void main(String... a) throws Exception {
    String E = a.length > 0?a[0]:"Joe Nartca";
    //
    String X = EnDecrypt.encrypt(E);
    System.out.println("Given:"+E+"\nEncrypted:"+X+"\nDecrypted:"+EnDecrypt.decrypt(X));
  }
}
The Encrypting and Decrypting is dynamic. If you repeat the TestEnDecrypt you could see the encrypted string "Joe Nartca" is very, very rare the same.
TestEnDecrypt.png


VI. Object-Oriented DB Server Implementation

JAVA is a relatively old OOPL which was derived from C/C++ with the main objective of portability between different Operating Systems: WORA -Write Once Run Anywhere. Due to that OODB on the server site is implemented in JAVA.

Simplicity is the best implementation. The famous Russian inventor Mikhail Kalashnikov said “Things that are complex are not useful, Things that are useful are simple.” OODB is implemented with the greatest care of simplicity and efficiency (performance). The whole OODB package is composed of 19 APIs whereas 8 APIs are the APIs of ODB Client package. The ODB Server APIs are (all in JAVA):

1. EnDecrypt: the Encrypting/Decrypting Algorithm with two basic functions: encrypt and decrypt (server and Client)
2. ODBBroadcaster: the multicasting message sender as the Publisher (Server only)
3. ODBEventListener: the Message Listener as the Subscribers (server and Client)
4. ODBEventListening: the Interface to the Events (server and Client)
5. ODBEvent: the event object (server and Client)
6. ODBCluster: the intercommunicator between ODB Servers on a network (server only)
7. ODBConnect: the Interface of Client apps to ODB Server (Client only)
8. ODBInputStream: the input byte-stream similar to ByteArrayInputStream (server and Client)
9. ODBOutputStream: the output byte-stream similar to ByteArrayOutputStream (server and Client)
10. ODBSendStream: the extension of ODBOutputStream, similar to ObjectOutputStream (server only)
11. ODBReceiveStream: the opposite of ODBObjectStream, similar to ObjectInputStream (server and Client)
12. ODBParms: POJO of ODB for the intercommunication between ODB APIs (server only)
13. ODBParser: the Configuration File Parser (server only)
14. ODBService: the interface of Server implementation to OODB (server only). This API requires a Configuration file (see next page).
15. ODBManager: the ODB Data Management (read/write/lock/commit/etc. Server only)
16. ODBWorker: the interface between Client and ODBManager (server only)
17. ODMS: the Object Data Management System. The interface between ODB and the OS (as gzipped files to OS file system. Server only)
18. UserList: the User management System (create/delete/update user profile. Server only)
19. ODBMining: an extension of ODBConnect allows the clients to work with the ODB similar to SQL. Example: selectAll(dbName, salary, "EQ", 123000). An optional API of the Client API package

The jar files (without Debug-Information) are ~44 KB (Server) and ~18 KB (Client). All the sources will be included in an additional ZIP package -inclusive the EnDecrypt.java. However, it should be clear that the encrypting/decrypting algorithm should be only binary-known on the Client site. Otherwise, as aforementioned, such an encrypting/decrypting algorithm is useless when every public client “knows” how the encrypting/decrypting algorithm works. The IO Streams are ODB proprietary and specifically designed to enhance to IO throughput.

The syntactic descriptions of all ODBServer APIs are documented in Javadoc-style (generated by javadoc) and included in the ODBServer APIs package. The Server application is an application that must be separately developed around the API OBDService where the internal SERVER runs as an independent thread.

ODB_Network.png
The Client-Server Architecture (Cloud Computing)


ODBService of a Server Application requires a Configuration file which could be as following:
XML:
<! -------------------------------------------------------------------------------
   this is an example (Case Sensitive)

   - ODB_PATH is the root directory for all ODB
   - LOG is the name of the logging file (abs.path)
   - MAX_BUF is the send/receive Max Buffer
   - USERLIST is the encrypted file of all odb users
   - PRIMARY is the acting server
   - NODE_X is the cluster at node X
   - WEB_HOST/IP is the WEB name or IP of the Computer Node.
     localhost is the web name if it's local only

   Convention: Host and port must be separated by a colon (:) If IP is given
               it must be in binary form (separator is a dot, NOT a colon (:)
               even with IP6)
---------------------------------------------------------------------------------
   Primary Server
!>
<WEB_HOST/IP>localhost />
<PRIMARY>9999 />
<! -------------------------------------------------------------------------------
   Node Servers
   Format: WEB_HOST/IP:Port
   clustered Servers: omitted if no clustered servers
   WEB_HOST/IP: see above.
   Convention index: _1, _2, _3. ... for the Node No.1, No.2, No.3, ...
!>
<NODE_1>localhost:8888 />
<! -------------------------------------------------------------------------------
  Messaging facility for the nodes using the Multicast IP
  The multicast datagram socket class is useful for sending and receiving IP multi-
  cast packets. A MulticastSocket is a (UDP) DatagramSocket, with additional capa-
  bilities for joining "groups" of other multicast hosts on the internet.
  More about MulticastSocket: see MulticastSocket API
  About Global Internet Multicast IP:
     http://www.tcpipguide.com/free/t_IPMulticastAddressing.htm
!>
<MULTICASTING> 224.0.1.3:7777 />
<! -------------------------------------------------------------------------------
   working environment
!>
<ODB_PATH>C:/JoeApp/ODB/Nodes/oodb_Node1 />
<LOG>c:/JoeApp/ODB/Nodes/log_Node1/log.txt />
<USERLIST>c:/JoeApp/ODB/Nodes/oodb_Node1/userlist />
<! ----------------------------------------------------------------------------- !>
Syntax of Configuration file (based on XML syntax):
  • comment: always starts with <! and ends with !>
  • keyword: always embedded by < and > followed by its value and closed by />
and it must contain the following entries:

1. Primary Server with WEB_HOST/IP for WebHostName or WebHostIP and PRIMARY for the port number
2. An optional list of nodes starting with NODE_1 … NODE_X containing the string value WebHostName/IP : Port
3. The Broadcaster or Publisher MULTICASTING with the value multicastIP: Port
4. ODB_PATH is the path where the Object-Oriented databases are stored
5. LOG is the file (abs. Path) where the logging can be made
6. USERLIST is the file where the list of authenticated users.

Example: a simple Server Application
Java:
import joodb.ODBService;
// Joe Nartca (C)
public class SimpleServer {
  public SimpleServer(String config) throws Exception {
    new ODBService(config);
  }
  public static void main(String... argv) throws Exception {
    new SimpleServer(argv[0]);
  }
}
Suppose that odbConfig_Node1.txt is the configuration file for the node 1 the invocation is as following:
Code:
C:\ODB\Nodes>java SimpleServer odbConfig_node1.txt
ODBService is a Thread extension whose task is to load the configuration data, to start the ServerSocketChannel and to instantiate ODBManager. As soon as ServerSocketChannel receives a Client connection request it immediately launches an ODBWorker as the Peer to the Request Initiator which is either an ODBConnect client or an ODBCluster agent.

Depending on the number of ODB and the size of each an ODB Server with ODBService & ODBManager & ODMS requires a starting heap size of min. 4 GB for the Java Virtual Machine (JVM). It is recommended that ODB Server should work in an environment of abundance memory (up 16 GB).

ODBService offers the following methods that enable the users to build a GUI Server app in SWING or JavaFX (JAVA Syntax):
Java:
activeClients() returns a list of string containing all active clients who connect to the Server. The “client” is a concatenated String of userID, @ and 7 hes digits.
Synopsis: ArrayList<String> activeClients()

activeWorkers() returns a list of string containing all active clients who work directly with the specified dbName. The “client” is a concatenated String of userID, @ and 7 hes digits.
Synopsis: ArrayList<String> activeWorker(String dbName).

addCluster() adds the given node to the cluster. Where node has the format: WebHostName or WebHostIP:port
Synopsis: void addCluster(String node)

removeCluster() removes the given node to the cluster.
Synopsis: void removeCluster(String node)

broadcast() broadcasts (publishes) a message to all subscribers.
Synopsis: void broadcast(String msg)
Note: Following messages are reserved (case sensitive) and shouldn't be used:
1. WebHostName/IP: Port is OFFLINE or WebHostName/IP: Port is ONLINE
2. WebHostName/IP: Port is DOWN or WebHostName/IP: Port is READY
3. WebHostName/IP: Port is online
Java:
forcedClose() forces to close dbName of the specified node.
Synopsis: void forcedClose(String node, String dbName)
forcedFreeKey() forces to free the specified key of dbName.
Synopsis: boolean forcedFreeKey(String dbName, String key)
forcedFreeKeys() forces to free ALL key of dbName of the specified node.
Synopsis: void forcedFreeKeys(String node, String dbName)

forcedRollback() forces to rollback all keys of dbName of the specified node.
Synopsis: void forcedRollback(String node, String dbName)

forcedRollbackKey() forces to rollback object with the specified key of dbName.
Synopsis: boolean forcedRollbackKey(String dbName, String key)

ownerKeyList() returns the owner key list of dbName.
Synopsis: ArrayList<String> ownerKeyList(String dbName)

ping() ping a node and returns a time in milliseconds
Synopsis: long ping(String node)

register() register to this local Publisher.
Synopsis: void register(ODBEventListening listener)

shutdown() shutdown ODBService gratefully.
Synopsis: void sutdown()
An ODBService Application in SWING left) and JavaFX (right)
SWING_JFX.png

UserList is another API. This API is the foundation for a successful connection to an ODB Server. ODB users can only access the Object Databases when they are registered in conjunction with a Privilege which is 0 (Read only) or 1 (Read-Write) or 2 (Read-Write-Delete) or 3 (2 with Superuser or su Privilege). UserList encrypts registered userID&PW and saves them in the GZIP file userlist. The JFX server applies this API: see Tab UserMaintenance.
Java:
addUser() adds a new user with the specified Privilege to the userlist
Synopsis: boolean addUser(String pw, String uid, int priv)
Return: true – new user is added successfully
deleteUser() deletes a new user from the userlist
Synopsis: boolean deleteUser(String uid)
Return: true – new user is deleted successfully

save() saves all modification
Synopsis: void save( )

getUserList() returns a list of all users. The parameters pw and uid are the Superuser PW and Superuser ID. The format of an entry of the list: PW:UID
Synopsis: ArrayList<String> getUserList(String pw, String uid)

getUserPrivilege() returns the privilege of the specified user
Synopsis: int getUserPrivilege(String pw, String uid)

isSuperuser() is the given PW & uid a superuser
Synopsis: boolean addUser(String pw, String uid)

isUser() is the given PW & uid a valid user
Synopsis: boolean addUser(String pw, String uid)

changePassword() changes the PW of an user
Synopsis: boolean updateUser(String uid, String pwOLD, String pwNEW)

updatePrivilege() updates the Privilege of an user
Synopsis: boolean updateUser(String suPW, String suID, String userID, int priv)

resetPassword() allow the client to change the password
Synopsis: String resetPassword(String uID)
Server_UserList.png
The UserMaintenance Tab
 
Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
2,927
1,312
113
The delivery ZIP package JOODB contains the default userlist in directory Node_1 with the default Superuser whose Password is system and ID is admin (both in lowercase). In case that the superuser password is somehow forgotten the Recovery reconstitutes the Superuser system/admin while the entries of other users are unaffected.

How does the Client/Server architecture work?

When an ODB Server is started it firstly begins to inform (ODBBroadcaster) its online status to all other ODB Servers on the net (ODBEvent/ODBListening/ODBListener) and establishes a connection to each of them (ODBCluster) so that all ODB Servers in the Cluster know who of them is online or offline. It is an INTERCONNECTED NETWORK. One to All and All to ONE.

When a Client starts to connect to any of the ODB Servers (ODBConnect) in the cluster and if the connection is successful the connected ODB Server immediately spawns an ODBWorker thread that designates to work directly with this Client app (ODBConnect). Then when the Client starts to open an Object Database (connect command) ODBWorker delegates the task to ODBManager to work out the request. ODBManager starts its own request to all other online ODB Servers in the cluster (ODBCluster) to provide the parts of the Request if these parts do not locally exist. For the Client this interconnecting operation is invisible. What the client sees is an Object Database as the whole (Distributed Object Database). ODBWorker always works directly with a client (ODBConnect) and several cluster agents (ODBCluster).

Client access to a certain object occurs locally from its viewpoint, even the requested object has to be internally delegated to one of the ODB Servers in the cluster where the object exists (via the local ODBCluster which communicates with the remote ODBWorker of this remote ODB Server).

Due to the complexity of network-wide management and the persistence of distributed Object data it is currently intended that Clients or users up Privilege 1 (incl. Superuser) can only create a new Object Database or a new part of a distributed Object Database locally. The visible advantage is in its nature: only the local DB Administrators know the best and usually understand the local computing environment. Further, remote administrators could cause real havoc to the databases and no one knows where and how the damages occurred.

Multiple Object Databases – Single Connection

With a single ODBConnect Client application can access to several Object Databases (ODB) at a time. With the method connect(dbName). With the distinctive dbName every ODB can be individually accessed. Furthermore: each ODB can consist of different (serialized) objects which can be not only different in size, but also in data structure and in processing methods.

Because each object has its own processing methods and data they all are accessed by their distinctive key (or name). With the key a serialized object class can be classified, its methods can be identified and reflectively executed. All that is implemented and realized in ODBMining, an extension of ODBConnect. To facilitate the accesses to ODB object by field names (variables) ODBMining provides some selectAll() methods which are similar to SQL-SELECTS (more about the methods: see the incl. ODBMining JAVA-Doc style).

A Zoo is an example for a combination of different and distinctive objects within an Object-Oriented Database: Building structure and architecture, staff and the animals which are also distinctive (plants, herbivores, carnivores, fishes, insects, etc.) These objects can be separately grouped in an ODB and can be accessed separately or together.
Example: The HighSchool app works with ODB highschool and it has FIVE different objects:
  • Aid
  • Student
  • Teacher
  • Admin
  • School
school.png

Let look at the Aid object:
Java:
import javax.swing.JFrame;
public class Aid implements java.io.Serializable {
  private static final long serialVersionUID = 1234L;
  public String name, pic, job, school;
  public boolean gender; // true: male, false: female
  public double salary;
  public Aid(String name, String pic, String job, String school, boolean gender, double salary) {
    this.name = name;
    this.pic = pic;
    this.job = job;
    this.school = school;
    this.gender = gender;
    this.salary = salary;
  }
  public String toString() {
    return (gender?"Mr.":"Mrs.")+name+" (Pic."+pic+") works at "+school+" as "+job+
           ". "+(gender?"He":"She")+" earns "+salary+" US$";
  }
  public void show(JFrame jf) {
    new Pic(jf, pic, name);
  }
}
And the access to Teacher Joe with reflective execution of toString() and show()
school_1.png

and something similar “table with rows and columns”
school_2.png

or only with Object Teacher
school_3.png

The Client package: ClientODB.zip contains the following directories and files:

2) Text files:
- readMeFirst.txt tells you how to work with the ODB package
- intro.txt for the example ZOO
2) JAR files:
- joodb.jar: the ODB package (Client + Server)
- clientODB.jar: the Client package
- joemvc.jar: the SWING-MVC package used by JODBServer and the People-Example (see HERE)
- JODBServer.jar: the executable SWING ODB Server
- JfxODBServer.jar: the executable JFX OBD Server
- jlayer-1.0.1.jar: The JavaZOOM's MP3-player
3) Bat-files
- build.bat is the batch file to build the clientODB.jar
- setCLASSPATH.bat is to set CLASSPATH and PATH before you can run any mentioned Server JAR or examples
4) JAVA Sources ODBCore (Client only)
- EnDecrypt.java
- ODBConnect.java
- ODBEvent.java
- ODBListener.java
- ODBListening.java
- ODBInputStream.java
- ODBMining.java
- ODBOutputStream.java
- ODBReceiveStream.java
5) Nodes
- log_Node1: contain the log-file of Node 1
- log_Node2: contain the log-file of Node 2
- odb_Node1: contain the OO databases of Node 1
. eDict: The Vietnamese-English Dictionary (part 1)
. people: the People database (part 1)
. HighSchool: the HighSchool database
. userlist: the userlist with 2 predefined users: superuser admin/system and test/tester (ID/PW)
. zoo: the ZOO database.
- odb_Node2: contain the OO databases of Node 1
. eDict: The Vietnamese-English Dictionary (part 2)
. people: the People database (part 2)
6) doc (as APIs_Doc.zip)
- containing all JAVA-Doc-styled html-APIs decription
7) Examples
. eDict: the sources and icons direcrory
. people: the sources and the SWING-MVC file
. school: the sources
. zoo: the sources.

The ZOO
Run_Package.png

or ThePeople
Run_Package_1.png

incl. ClientODB.zip

Node: the 2 ZIP ExamplesZOOSounds_1 and _2 belong to the ZOO subdirectory sounds.

If you have questions about ODB packages (Client & Server) you shoudn't hesitate, but inbox me with your questions or requests.
 

Attachments

Sửa lần cuối:
  • Like
Reactions: ngtheanh.dev