O O D B Design And Implementation - Part 4 -

Joe

Thành viên VIP
21/1/13
2,969
1,310
113
Hi

(cont. of Part 3)

OODB Infrastructure on the Server site
In Section 3 I have discussed about the infrastructure on the Client Site. On the Server site it's more complex and trickier. On the Server site we have to determine an environment that must be secure, but open for all kinds of clients. A contradiction in itself.

The most conventional, but most approved method is the authentication with Password (with a min. length of 5 letters) and UserID (which is unique). However, if the server is one of N servers or Nodes in a cluster where each node has its own local users the uniqueness of UserID is a real problem. And we cannot demand the users to memorize a long, unfamiliar ID. Here we have to provide a dynamic mechanism that makes the UserID distinct among themselves and independent from the Nodes.

Because each local Node owns its own UserList of users who could share the same UserID with some other users of another Node, but they have different accessing privileges (see Section 3) we have to make sure that they won't get in their way with each other privilege and cause some inconsistencies of OO data (see distributed DB in section 2). This safeguarding requirement is the fundament of an infrastructure on the Server site.

And I have briefly talked about the UserList and its basic requirements: the different privileges and the Superuser feature. The Superuser has the highest privilege (2) and the power to update or to delete an user out of the UserList. The delivered UserList contains only one default Superuser with the Password "system" and the ID "admin" (both in lowercase). However, for security reasons this default Superuser is barred from accessing to the core of ODB services. The ODBWorker which is responsible for the interfacing between user/ODB keeps an eye on this default Superuser and denies him any further forthcoming. Therefore you or other ODB administrators should create for yourself/themselves your/their own Superuser.

In a "messy" environment it's always hard to manage. The ODB server site should divide into some (clean) areas and clearly organized so that it's easier to maintain. In my view an ODB server site should be separated at least into three folders (or directories):
  1. Folder joodb contains the ODBs. All OO data are stored in this folder under its own ODB name. Example: poeple, employees, etc. Because the UserList is in itself an ODB it should be stored in this folder, too.
  2. Folder log contains the daily logs.
  3. Folder Node contains the executable jar of ODB Server, its dependent external jars (e.g. joemvc.jar if you use my provided JOODBServer) and the configuration files (oodbConfig.txt)

JOODBServer

UserList Maintenance

JOODBServer can be started by everyone, but only the Superuser can shutdown it, or has accesses to the UserMaintenance Tab.

The ODB package is a bundle of APIs that constitute the ODB core. To have a full-fledged ODB Server environment you have to develop you own application using (wrapping-around) the API ODBService of this ODB core (see the 1st picture of section 2).

Note: ODBService is a [/b]self-starting[/b] Thread.

The counterpart of ODBConnect is on the Server site ODBWorker. As said, both communicate to each other via SocketChannel. The transferring OO data are either "plain" or in GZIP data. Data are GZIP only when their size is larger than 256 bytes (otherwise more bytes are generated than the original size). This dynamic communication (see HTTP/HTTPS) relieves the traffic load and in general reduces the latency time on the net(better response time). The drawback of such an "extra" computing work is less than some microseconds so that it can be ignored (see the coding snippet), but the transferring of extra bytes could take more times on the net than the computing time on both sites (client and server).

PHP:
  ...
  public void gzip(SocketChannel soc, byte[] buf) throws Exception {
    ByteArrayOutputStream bao = new ByteArrayOutputStream(buf.length);
    GZIPOutputStream go = new GZIPOutputStream(bao, true);
    go.write(buf);
    go.flush();
    go.close();
    //
    buf = bao.toByteArray();
    if (buf.length <= 32768) {
      soc.write(ByteBuffer.wrap(buf));
      return;
    }
    for (int p = 0, le = 32768;; p = le) {
      soc.write(ByteBuffer.wrap(buf, p, 32768));
      le += 32768; // next block
      if (le >= buf.length) { // last portion
        soc.write(ByteBuffer.wrap(buf, p, buf.length-p));
        return;
      }
    }
  }
  ...
The following protocol shows the dynamicism of "GZIP on Demand" (only if the size is larger than 256 bytes). It reduces the latency time and relieves the traffic load on the net.
Code:
C:\JoeApp\ODB\Node_1>go
Size before send:363-->GZIP & send:322
Size before send:410-->GZIP & send:352
Size before send:387-->GZIP & send:329
Size before send:302-->GZIP & send:281
Size before send:302-->GZIP & send:281
Size before send:302-->GZIP & send:281
Size before send:302-->GZIP & send:281
Size before send:492-->GZIP & send:366
Size before send:302-->GZIP & send:281
Size before send:302-->GZIP & send:281
Size before send:302-->GZIP & send:281
Size before send:302-->GZIP & send:281
Size before send:480-->GZIP & send:355

C:\JoeApp\ODB\Node_1>
The ODB Configuration
ODB core requires some external inputs for proper startup: the paths to the databases and UserList, the ThreadPool size for ODBWorkers, the Server IP or HostName and its Port, etc. All these inputs must be formulated in XML style. And the syntax is relatively simple (Kalashnikov):
  • Comment starts with <! (less+exclamation mark) and ends with !> (exclamation mark+greater).
  • Keyword is embedded by < and >
  • Value in-between and closed by /> (slash+greater)

Example:
PHP:
<!-------------------------------------------------------------------------------
   this is an example (Case Sensitive)
   - ODB_PATH is the root directory for all ODB
   - LOG is the name of the logging file (abs.path)
   - MAX_BUF is the send/receive Max Buffer
   - POOL gives the max number of active threads in the ThreadPool
   - USERLIST is the encrypted file of all odb users
   - PRIMARY is the acting server
   - NODE_X is the cluster at node X
   Convention: Host and port must be separated by a colon (:) If IP is given
               it must be in binary form (separator is a dot, NOT a colon (:)
               even with IP6)
---------------------------------------------------------------------------------
   Primary Server
!>
<PRIMARY>localhost:9999/>
<!-------------------------------------------------------------------------------
   clustered Servers: omitted if no clustered servers
   Convention index: _1, _2, _3. ... for the Node 1, 2, 3, ...
!>
<NODE_1>localhost:9980/>
<!-------------------------------------------------------------------------------
  Messaging facility for the nodes using the Multicast IP
  More about Multicasting Socket:
  Read this: http://www.steves-internet-guide.com/introduction-multicasting/
!>
<MULTICASTING>224.0.0.3:8888/>
<!-------------------------------------------------------------------------------
   working environment
!>
<POOL>512/>
<ODB_PATH>C:/JoeApp/ODB/Node_1/oodb_1/>
<LOG>c:/JoeApp/ODB/Node_1/log/log.txt/>
<USERLIST>c:/JoeApp/ODB/Node_1/oodb_1/userlist/>
<!-----------------------------------------------------------------------------!>
Infrastructure Summary
Before we start to design and to implement our ODB let's summarize our requirements.

Client
  • Authentication with Password/UserID and Working privilege (0, 1, 2)
  • Plug-In API with the basic DB funtions (here: ODBConnect)
  • Extension Possibility (e.g. ODBMining)
  • Modern networking (SocketChannel)
  • Dynamic Data compression (GZIP)

ODB Server
  • Authentication facility with Superuser feature
  • Functional ODB core with Distributed ODB characteristics
  • Modern Networking (SocketChannel)
  • Dynamic Data compression (GZIP)
  • Clustering architecture
  • Auto-Recognition of ONLINE and OFFLINE Nodes
  • Auto-Connection and Disconnection with the late-coming or shutdown nodes
  • Easy Configuration and maintenance (e.g. odbConfig.txt in XML-Syntax)
  • Multiple User-Environment

(Next: OODB Design and Implementation)
 
Sửa lần cuối:
  • Like
Reactions: Thanhpv and quydtkt