Create a Project based on P2P/PubSub with OODB and DataMining - DataMining

Joe

Thành viên VIP
21/1/13
3,007
1,335
113
(Continue of HERE)

Create a Project based on P2P/PubSub with OODB and DataMining - Data Mining

You may wonder where do the Subscribers intercept the published messages? In the starting app Boutique (main program). Depending on the user profile as Salesperson, external online clients, staff of Order or Billing or Accounting or HR or Managing department Boutique appearance is then individual, but all shares the same common POJO Parms and the initialization process. In here MessageSender (Publisher) and MessageListener (Subscriber) are started in a common ForkJoinPool (see HERE for the reasons). While the Sender is accessible directly by Publishers the Listener works autonomically together with its assistant ModelWorker who then works out with the selective tasks (update, add, delete or ignore).

The main App Boutique.java
Java:
// OODB
import joeapp.odb.*;
// Joe Nartca (c)
public class Boutique extends JFrame {
  public Boutique(String[] args) throws Exception {
1     P.frame = this;
      try {
2       map = ODBParser.odbProperty(args.length > 0 ?args[0]:"boutique_config.txt");
      } catch (Exception ex) {
        JOptionPane.showMessageDialog(this, "Unknown Config", "Alert", JOptionPane.ERROR_MESSAGE);
        ex.printStackTrace();
        System.exit(0);
      }
      setTitle("Boutique");
3     JTabbedPane tabs = new JTabbedPane();
4     tabs.setPreferredSize(new java.awt.Dimension(1000, 500));
      getFrames()[0].setIconImage(java.awt.Toolkit.getDefaultToolkit().
                                  createImage(ClassLoader.getSystemResource("ShoppingCart.png")));
      ...
      // PubSub port & IP
5     P.port = Integer.parseInt(map.get("MESSAGE_PORT"));
6     P.host = map.get("MESSAGE_SERVER");
7     listener = new MessageListener(P);
8     P.sender = new MessageSender(P);
9     P.pool.submit(listener);
10    P.pool.submit(P.sender);
      ...
}
Explanation:
  • Line 1..4: initialize the App with JFrame and JTabbedPane
  • Line 5..6: get the HostName and Port from the file Boutique_Config.txt
  • Line 7..8: instantiate MessageSender and MessageListener (both sources in THIS session, incl. the source of ModelWorker and to learn how ModelWorker works)
  • Line 9..10: start them in the ForkJoinPool

DataMining

Today I talk about the Data Mining from the OODB Invoice and Item. Even I have performed only some transactions, but the data are sufficient enough to show you how to analyze the data for the hidden information and how to visualize the results so that the higher management could understand, and as well as the laymen.

First: the full source of POJO Invoice that you can find it in the THIS session. Then the source of POJO Item:
Java:
// Joe Nartca (c)
public class Item implements java.io.Serializable {
  private static final long serialVersionUID = 1234L;
  /**
  Constructor
  @param itemName String, item name
  @param price    double, selling price
  @param cost     double, net cost price (purchasing, marketing, production)
  @param stock    int, items in stock
  @param description String, item description
  @param picture String, link to Image or Image FileName
  **/
  public Item(String itemName, double price, double cost, int stock, String description, String picture) {
    this.cost = cost;
    this.price = price;
    this.stock = stock;
    this.itemName = itemName;
    this.description = description;
    this.picture = picture;
    nStock = stock;
  }
  public String toString() {
    return "Item Name: "+itemName+", Price: "+price+", In Stock: "+stock+", Description: "+description;
  }
  public String[] toArray() {
    return new String[] { itemName, ""+price, ""+stock, description, picture };
  }
  public String asItem() {
    return itemName+", Price:"+price+" US$ ("+description+")";
  }
  public void setPrice(double p) {
    price = p;
  }
  public void setStock(int n) {
    stock = n;
  }
  public int updateStock(int n) {
    int order = stock - n;
    if (n > stock) stock = 0;
    else stock -= n;
    return order;
  }
  public void setCost(double c) {
    cost = c;
  }
  public double getPrice( ) {
    return price;
  }
  public int getStock( ) {
    return stock;
  }
  public int getDefaultStock( ) {
    return nStock;
  }
  public String getItem() {
    return itemName;
  }
  public double getCost( ) {
    return cost;
  }
  public String getDescription( ) {
    return description;
  }
  public void setDescription(String desc) {
    description = desc;
  }
  public void setImage(String img) {
    picture = img;
  }
  public String getImage( ) {
    return picture;
  }
  public boolean equals(Item I) {
    return (itemName == I.itemName && price == I.price && cost == I.cost &&
            stock == I.stock && description == I.description && picture == I.picture);
  }
  //
  private int stock, nStock;
  private double price, cost;
  private List<Integer> sold = new ArrayList<>();
  private String itemName, picture, description;
}
Both POJOs are the plain Java Objects with some getters and setters. Both are serialized POJOs and the accessing key is the Invoice date (ODB Invoice) or Item Name (ODB Item). The Accessing and Storing of Invoice and Item are done with OODB package (see HERE). For that we need an OODB Server and an OODB Client package.
  • OODB Server: either in SWING (JODBServer.jar) or JavaFX (JfxODBServer.jar) and the ODB server package (jodb.jar).
  • OODB Client: clientODB.jar (used by Boutique)
All the JAR files will be bundled together with the Boutique sources and posted at the end of this series. The access to the OODB Invoice, Item (and HumanResource as well) is done by the module OpenOODB. A brief coding snippet of OpenOODB shows you how the serialized data are interchanged with the OODB Server in P2P communication with SocketChannel:
Java:
import joeapp.odb.*;
// Joe Nartca (c)
public class OpenOODB {
  /**
  @param uid    String, user ID
  @param pw     String, user Password
  @param host   SODB Server name or IP
  @param port   int, the port number
  @param sorted boolean, true if the keys should be sorted
  */
  public OpenOODB(String uid, String pw, String host, int port, boolean sorted) throws Exception {
    odbc = new ODBConnect(host, port, pw, uid); // SocketChannel Communication
    this.sorted = sorted;
  }
  public boolean connect(String dbName) {
    try {
      odbc.connect(dbName);
      odbc.autoCommit(dbName);
      dbList.add(dbName);
      return true;
    } catch (Exception ex) { }
    return false;
  }
  public String[] getKeys(String dbName) {
    List<String>iList = sorted? odbc.getSortedKeys(dbName):odbc.getKeys(dbName);
    done = (iList != null);
    if (!done) return null;
    return iList.toArray(new String[iList.size()]);
  }
  ...
  public void lock(String dbName, String name) {
    try {
      done = odbc.lock(dbName, name);
    } catch (Exception ex) {
      done = false;
    }
  }
  public void unlock(String dbName, String name) {
    try {
      done = odbc.unlock(dbName, name);
    } catch (Exception ex) {
      done = false;
    }
  }
  public boolean update(String dbName, String name, Object obj) {
    try {
      odbc.update(dbName, name, obj);
      odbc.commit(dbName, name);
      done = true;
      return true;
    } catch (Exception ex) { }
    done = false;
    return false;
  }
  public boolean delete(String dbName, String name) {
    try {
      odbc.delete(dbName, name);
      odbc.commit(dbName, name);
      done = true;
      return true;
    } catch (Exception ex) { }
    done = false;
    return false;
  }
  public void add(String dbName, String name, Object obj) {
    try {
      odbc.add(dbName, name, obj);
      odbc.commit(dbName, name);
      done = true;
    } catch (Exception ex) {
      done = false;
    }
  }
  public Object read(String dbName, String name) {
    try {
      done = true;
      return odbc.read(dbName, name);
    } catch (Exception ex) { }
    done = false;
    return null;
  }
  ...
}
In real life Precious Metal Mining is the technical way how to extract the metal out of dirt and soil. Like that Data Mining is the art how to make something exciting out of all the mundane things. Let me show you an example. You get a bundle of invoices with different details:
  • number of purchased or ordered articles PER transaction
  • the paying type: cash, bank card, credit card
  • the names of the bank or credit card
  • online clients or direct sales
  • if direct sales who is the salesperson
  • etc.
Now you have to present the gathered data to the executives and make them understanding without being "grilled" with questions by them. Your career is highly dependent on such a presentation. Back to precious metal mining. If you present the executives several trucks full of gray soil, then you try to tell them that the trucks are full of money. They would stare at you and leave without a word. And the next day you are in real troubles. Similar to that, if you show the executives a batch of documents full of mundane words they firstly stare at the batch and then at you. The next day you are in troubles, too.

The art of Data Mining starts with the design of the data structure and ends with the extraction of the data in the right details. If your data structure is badly or insufficiently designed your data get lost in the badness or insufficiency. If you are thrown into the middle of a work full with bad or insufficient design the first thing you have to do is to find out the badness and insufficiency and then analyze them whether they are somehow presentable or per se unusable. Otherwise you could be wrongly blamed as an unqualified person or developer.

The Design of Data structure should be as simple and as plain as possible and think about the famous say of master Kalashnikov about the useful simplicity over the useless complexity. Reasons? The more complex a data structure is the harder it is for an extraction of the useful details. And that could brings you into an inescapable maze of complexity. I show you the Data Structure of Invoice (see the source HERE). The Invoice Data Structure contains 3 substructures which are:
  • Ordering Data Information
  • Billing Data Information
  • Accounting Data Information
With the mentioned data it is possible to visualize and to present the Invoice Data. WYSIWYG -the famous say of old days: What You See Is What You Get. What the executives see is that what they get and...remember. Exactly as the ancient Philosophe Confucius said:
If you explain it to me, I'll forget it later.
If you show it to me, I'll remember it later
if you let me do it, I'll understand it.
statistics.png

statistics_profits.png

or about a certain client
billy.png

(continue HERE)
 
Sửa lần cuối: