Big Data and Data Warehouse

Joe

Thành viên VIP
21/1/13
3,074
1,344
113
Two Voodoo buzzwords. Before I give you a "special tutorials" with Implementation example in PURE JAVA, and without and 3rd Party products such as IBM MQ, nor Rabbit MQ let study the two following confusing pictures:

First, a glimpse at Big Data:
1629383237511.png
(Source: miro.medium.com)

Got it ? Then at the next Voodoo magics:
1629383449652.png
(Source: databricks.com)

Confused? Don't be shy and won't admit it. The two images fool and befuddle me, too. A lot of buzzwords. We, people and in particularly Americans, love to live in the virtual world with a lot of buzzwords that confuse and befuddle the others as much as possible. It makes us looking "special". Very special.

ERP? CRM? ETL? And so on.

Nobody knows exactly what they are and what they stand for. Every IT guru understands them under his or her own like-and-grudge. And to our worst: the wanna-be global companies blindly follow the trend and demand the applicants to master such Voodoo magics. Even the bosses of these "global companies" don't know a minute thing about that what they are demanding from the applicants. They fool and befuddle themselves together with the applicants.

The last Job offer from " Tuyển dụng nhân sự CNTT - IT recruitment" demands from the applicants to master IBM MQ and/or Rabbit MQ and so on. For what? Not a word about the MQ purposes. Again a nebulous buzzword that stands for "Message Queueing" (and invention of IBM in the '90 -quite ancient, isn't it?). When you behold the two confusing images, then concentrate yourself before you start to focus only on the essentials: the communication between the Voodoo sandboxes and you could get it. It's about the Messages. Well, when a huge bundle of incoming messages starts to overwhelm the one of the sandboxes some messages could get lost... Big problem: the loss. To cover the loss problem the (incoming) messages must be somehow queued for the later processing. The way how the message is queued is the birth of the buzzword MQ.

What is really a "Data Warehouse"? If you ask a drugstore owner about the number of articles he or she has in the shop then you could get a ton of information, or in other words: data, about each article and where it is purchased and what price it should be sold so that he or she could thrive, and then how the relationship between owner and producers and his or her own customers must be nursed, etc. Such the data are the BIG DATA of a Data Warehouse.

Data warehouse is always specific and should be developed individually. A generic form of Data Warehouse won't exist even among the companies with the same business. So, Big Data are the unrelated things and the huge hodge-podge of data is the Data Warehouse. To master such data it requires you a good instinct of the differences and of the sameness. And that could be worked out satisfactorily with distinctive Objects using OOP and OODB (Object Oriented Programming and OO Database). I show you step by step how to implement a solution for Big Data and Data Warehouse in PURE Java. However, the readers should be advanced and experienced in JAVA. Following preconditions should be installed on your computer:

- OODB (see: HERE)
- SWING MVC (see: HERE)
- Up JAVA 8. Better with JAVA 9 (because of JavaFX which is still be a part of JAVA)

To download the most recent releases: joodb.jar (Server site), ClientODB.jar (client site) and joemvc.jar
(will continue)
 

Attachments

Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
3,074
1,344
113
The boring chore:
1) unzip the oodb.zip to a directory of your choice. The Zip content: 4 folders: SWING, odb_Node1, log_Node1 and JFX
  • Folder SWING contains 5 files: g1.bat (to start the SWING OODB server), JODBServer.jar (the SWING server), joemvc.jar (the SWING MVC model), joodb.jar (the OODB) and oodbConfig_Node1.txt (the config file for the server)
  • Folder oodb_Node1 contains 4 databases: HumanResource, Items, Invoices and userlist (Server's login userlist)
  • Folder log_Node1 contains the file log.txt (logging Server activities)
  • Folder JFX is similar to Folder SWING, but the css file joe.css instead of joemvc.jar
2) set the global CLASSPATH to the mentioned jar files

Folgende users are preset in the userlist:
Code:
+--------------------------------+---------------------------------------+-----------------------------------------+
|        userID                  |            Password                   |         Privileg                        |
+--------------------------------+---------------------------------------+-----------------------------------------+
|        admin                   |            system                     |    Server superuser                     |
+--------------------------------+---------------------------------------+-----------------------------------------+
|        ceo                     |            chief                      |   DB & Server superuser                 |
+--------------------------------+---------------------------------------+-----------------------------------------+
|        beth                    |           beth                        |         Billing                         |
+--------------------------------+---------------------------------------+-----------------------------------------+
|       elli                     |            elli                       |         Purchasing (order)              |
+--------------------------------+---------------------------------------+-----------------------------------------+
|       david                    |           david                       |         Human Resource                  |
+--------------------------------+---------------------------------------+-----------------------------------------+
Note:
  • Only the superuser can manage the OODB Server, adds, deletes and updates the others.
  • The default ID admin (PW system) works solely on the server. He cannot login into any OODB.
Here are the GUIs of OODB Server in SWING and JFX:

JFX_SERVER.png

SWING_SERVER.png

(continue to HERE)
 
Sửa lần cuối: