Toggle Theme Editor
Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Charcoal

Artificial Intelligence And Fuzzy Logic (ai + Fl)

Discussion in 'Java Update' started by Joe, 7/4/19.

  1. Joe

    Joe Thành viên VIP

    Artificial Intelligence and Fuzzy Logic

    1. Artificial Intelligence (AI)
    2. Fuzzy Logic (FL)
    3. Repository
    4. Some Examples
     
  2. Joe

    Joe Thành viên VIP

    1. Artificial Intelligence (AI)

    Artificial Intelligence
    AI is in plain English: man-made Intelligence. Exactly as in Vietnamese: Trí Tuê Nhân tao. The word Intelligence is here in English a bit imprecise. More precise it is in Vietnamese "Trí Tuê". And "Trí Tuê" is "Knowledge". Without knowledge there's no intelligence. The right use of knowledge and intelligence is then the Wisdom.

    What is intelligence? What is knowledge? The intelligence bases on knowledge. That's the matter of fact. The more knowledge accumulates, the more intelligence is increased. Human and animal as well gain knowledge by observation and deduction. A wrong deduction leads to memorized experience. And experience is the base of knowledge. Example, if we touched an unknown insect (observation) and were stung by it, and it hurt immensely (experience). From this unpleasant experience we deduct a knowledge that this insect is dangerous and we won't touch such an insect again. With this new knowledge we were able to share or to warn the others by telling them about the dangerousness of the similar insects ahead (intelligence).

    Knowledge is also an accumulative process of
    1. Observation leads to inquisitiveness.
    2. Inquisitiveness leads to experimenting.
    3. Experimenting leads to experiencing.
    4. Experiencing leads to learning.
    5. Learning leads to acknowledging.
    6. Acknowledging leads to knowledge
    7. Knowledge leads to intelligence.
    Intelligence is individual and personal. Even a pair of monozygotic twins doesn't share the same intelligence. It's the matter of fact that people whose knowledge could be the same draw very different conclusions. Therefore knowledge is an individual and personal deductive process. A process of 4-Nesses. The S.C.A.R.
    • Similarness, not Equalness,
    • Commonness, not Exactness,
    • Approximateness, not Resoluteness.
    • Relativeness, not Absoluteness
    And intelligence bases on Knowledge. The consequence is: "natural" intelligence bases on a deductive process, not on a designated process which always produces the same equality, the same exactness, the same absoluteness and the same resoluteness. Man-made intelligence is the reflection or the mirroring of natural intelligence: AI or Artificial Intelligence is also an active process.

    We have learned that a deductive process is individual and personal. AI is also individual and personal, too. A process of S.C.A.R.. Similarness, Commonness, and Approximateness Relativeness. Hence it's impossible and possibly wrong if we try to create an AI system that can be multiplied, and they all produce the very same results. The laws of S.C.A.R. are broken and we get a dumb (but somehow "shrewd") system. Not a real Artificial Intelligence system, but a passive AI System.

    AI Structure
    The on-the-IT-market available AI libraries such as Tensorflow, Matlab, etc. are therefore considered as the "AI automation tools" for a computerized, passive AI System. Because of the same "library" the computerized, passive AI System always produce the same decision under the same conditions. And that is against the law of individualism and personalism. If we want to build an active AI system we have to create an individual-personal system that could make decisions on its own knowledge, not on the knowledge that bases of a common-multiple-user-library. Otherwise we just have a "shrewd automat" that just follow and do exactly what the library dictates. The crash of Boeing 737 Max 8 in Indonesia (Lion Air) and in Kenia (Ethiopian Airline) is the result of a shrewd, but not intelligent automat. Boeing AI works with a sensor which delivers the needed data for a maneuvering decision. It works so far with the Equalness, Exactness, Absoluteness and Resoluteness that there's no individual and personal room for an own decision. And the results were fatal: the same crash under the same conditions by the same delivered data.

    What's next? Whatsoever AI is the reflection of human intelligence. And human brain is individual and centralized if you so will. Knowledge is different from irritation or stimulation. Irritation always triggers an according unconscious reaction. For example, if our foot hits an obstacle and we fall forwards we automatically push our arms with spreading hand forwards to cushion the (heavy) fall. Such a reaction requires a quick processing of data (fall forwards) and locally (arms, hands). Therefore our spinal cord is "stretched" along the back. On the other hand, knowledge, as we already know above, bases on conscious "calculation". And calculation requires power. Our brain is therefore the most powerful Central Computing Unit -the CPU (or Processor)- with abundant "built-in cache and memory. The CPU is itself a network of neurons (Neuron Network) or a network of processors if you so will.

    We human start to mimic our brain and build a "networked" computer. INTEL® CORE™ i9-9900K PROCESSOR is for example a network of 8 cores (each core is a 32-bits processor). But the computing power of our brain is still unmatched. Even with the most futuristic Quantum Computer. An "autistic child" can deliver a precise result of the most complex equation faster than any known supercomputer. With our powerful computing might we are able to store, to access and to process data instantly. The outcome is a decision, an action or whatsoever which is always the most actual. It's the deduction of knowledge. Knowledge must be existed (stored), available (accessible) and materializable (deductible).

    The most basic condition for AI is also the deductibility of facts (data) which are usually presentable in mathematical or physical descriptions. For example: A circle consist of uncountable number of referent points that have the same distance to an anchor point. A pentagon is a circle with 5 referent points, a Hexagon is a circle with 6 referent points, etc. The math. description of a circle is the number referent points and their distance to an anchor point (called the radius). The knowledge of a circle is the known referent points and the radius. The decision on a shape whether it is a circle or not bases on this math. knowledge. If an AI system needs to decide on a shape with 5 referent points that share the same distance to an anchor point whether this shape a circle or not it depends on the deduction of S.C.A.R. -Similarness, Commonness, Approximateness and Relativeness. A computerized passive AI System will say that the shape is NOT a circle, but a pentagon or a hexagon. And that is the fundamental difference between active AI and passive AI (computerized machine or shrewd automat).

    The S.C.A.R. laws allow dynamically to make a decision that bases on the most actual conditions. The dynamic is flexible and different from case to case.
    • Similarness: A pentagon or a hexagon resembles a circle drawn by a child. For an active AI system it's sufficient enough to deduct that the drawn shape is a circle. A passive AI system rejects such a deduction that simply bases on a resemblance.
    • Commonness: Because an infinite number of referent points is unachievable unknown shapes are recognized by their commonness. A pentagon or a hexagon share commonness: the distance of all referent points to the anchor point is the same.
    • Approximateness: the resemblance is more visible and more reliable if more referent points are sampled. Theoretically 2 referent points can be deducted as a circle with the anchor point in the middle of the line between the 2 referent points. Approximateness is crucial for a crucial decision (Boeing 737 Max 8).
    • Relativeness: the flexibility is the base of every decision. Any restriction could lead to rigidity and limits any possibility to construct a final decision in case of decision deadlock.

    The AI Unsolvable
    Man-made Intelligence is NOT natural Intelligence. AI bases on the Knowledge which is derived from human knowledge. Natural Intelligence is the accumulation of evolutionary knowledge. And this accumulation is invisible and inaccessible by human. It's a natural process whose evolutionary knowledge is stored over millions of years in the DNA strand.
    • Emotion is individual and personal. The rate of emotion cannot be scaled or weighted by any available mean. Every rating is always perfunctory. Never perfect. Hence AI cannot rate the feeling or gauges the emotion. The try of Volvo to monitor the face of the driver at driving and to deduct a conclusion of his moody state is deceptive because the intention of the driver to cheat is invisible for the monitoring cameras.
    • Extrapolatability is individual and personal, too. The seriousness of a sentence depends, for example, on the pitch of the speaker or the way how the speaker said. A loud you can be either a friendly warning from a danger or an unmistakeable contempt.
    • Interpretability is invisible and ambiguous. A polite rejection is usually very friendly, but uttermost unmistakable. In some cases a friendly YES is an unbridgeable, polite NO. Therefore AI-driven translator is always mediocre, never perfect (like human) and in many cases the translation turns out a nonsense.
    • Rationality and Irrationality are uttermost human. Decision depends on his temper or on some unknown reason (e.g. sympathy or apathy or hostility) and it can be very different on the very same matter. The prejudice is, for example, the most irrational decision of human. Why Vietnamese hate the Chinese so much ? Because China invaded and colonized Vietnam. Vietnam was invaded and colonized by France, too. But why Vietnamese don't hate the French as much as the Chinese? Or: why Chinese deeply hate the Japanese? Because Japan invaded and defeated China. China was invaded and defeated by Great Britain, too. But why Chinese don't hate the Brits as deep as the Japanese? It's the Prejudice. Prejudice cannot be mathematicalized or physicalized. And AI bases on Mathematics or Physics.
    • Imaginability is the most complex ability of human. The ability to see the unseen and to materialize it. For example, we imagine how "Demons", "Ghosts", etc. look like and then we sculpt them according to our imagination. It's impossible to formulate an imagination in any math. formula or to render it in any physical form. Religions are the most visible, the oldest existence of human imagination.
    • Fantasizability is also a very complex for AI-izing. Fantasy is the work of human brain which creates images or actions from nonentity. Dream or Fantasy is so individual and so personal that, for example, any description about it triggers the hearers their own dream or fantasy. Their individual and personal dreams or fantasies.
    • Learnability is the most unique ability of human (and animal). If a child got hurt by fire it quickly learns that fire is dangerous. Not the form of the fire that hurt it, but any form or appearance of fire. It's the learnability that allows human to extrapolate, to conclude and to deduct the impact.
    The complexity of human Intelligence is so tremendous that a Humanoid (man-made human-like robot) can and will never match human being. Can you imagine that a humanoid falls in love with another humanoid? I can't.

    AI Implementation

    AI Implementation is difficult. Very very difficult. Human knowledge is an accumulation of millions of evolutionary years of knowledge which is in most circumstances impossible to reflect or to describe in mathematics or in physics. Imagination or irrationality is, for example, the product of evolutionary knowledge, not a product of Mathematics or Physics. And without Mathematics or Physics AI becomes a coded sequence of dumb actions.

    Nevertheless there are a lot of human knowledge which can be solved by mathematics or physics. The Dijkstra algorithm is, for example, the search for the shortest path between two locations (or nodes) among numerous possibilities (locations). In reality we usually cut short our way in order to be faster and to lessen our effort (from walking). Dijkstra's Algorithm calculates the distances and decides which distances are the shortest so that the distance between the Begin and the Goal is the smallest sum of connected distances. This example demonstrates how Mathematics is applied to solve a human knowledge: cut short the (walking) distance. Nothing is fancy in Dijkstra math. algorithm. Only addition and comparison are involved. But the sequence of calculation-Comparison is decisive. It's the algorithm. AI is a sum of implemented algorithms. Our networked neurons (brain) possesses an unknown (or unlimited) number of sleeping algorithms which are waiting for us to be detected and to be woken up -like the Dijkstra Algorithm. The detecting base (of algorithm) is the 4-Nesses or S.C.A.R..

    The weighting of one of the S.C.A.R. determines how "specific" AI implementation should be.
    • Pattern Recognizing sets its weight on Similarness. Face or Fingerprint Recognition is the most typical AI-application which works intensively with physical resemblances and mathematical similarities.
    • Machine Learning works on the base of Commonness and Approximateness. ML could be either Supervisory or Unsupervisory Learning. A supervisory AI-Learning won't make the final deduction, but rather gathers the conditions that support the deduction which is then decided by the supervisor (i.e. user). An opposite is the Unsupervisory AI-Learning wherein it relies on its learned repository knowledge and makes the deduction from the learned knowledge by itself -without any intervention of its supervisor. The bad fate of Boeing 737 Max 8 could be the problem of self-deduction made by the passive Unsupervisory ML.
    • Gaming is the most widespread AI application. Gaming ranges from Chess Computer to (Car) Navigation System. The Relativeness decides more about the deduction than the Commonness or Approximateness or Similarness. Alpha-Beta Pruning Search or MiniMax-Search, for example in TicTacToe, deducts the best move (node) from all the available moves. Or MRA (Map Reduce Algorithm) is used by Google to search for the most relatively resemblant pattern within objects out of a heap of objects.
    • Warfare and Weapons Technology are the "most lethal" AI application. Herein all S.C.A.R. are applied. Kamikaze drones, smart torpedoes, intelligent missiles and so on are the last (and the stupidest) AI research and development. Unsupervisory AI missile finds its way to the goal based on its own repository knowledge and data fed by satellites and makes its own decision where to "seppuku" in order to create the most destructive result.
    However AI implementation requires a deep understanding of mathematics and a good imagination to visualize the (math) knowledge. The Cut-Short algorithm bases on two simple mathematical formulas: addition and comparison. Dijkstra's feat was the visualization of the Cut-Short knowledge with the 2 simple formulas. Simplicity is the key. Complexity is the dead end.

    [​IMG]

    AI Architecture

    AI Architecture is similar to our brain (as a single unit) which relies on
    • Powerful computing unit to execute complex mathematical paths (e.g. Dijkstra or MiniMax paths),
    • Plenty of memory to reposit the "gained" experiences (i.e. knowledge),
    • Reliable Input/Output units (as sensors like human eyes, ears, skin (for feeling), etc.),
    • Cross-Disciplinary working units to combine and to deduct an event in computable or representable form.
    Computing Unit (CU): Contemporary technology allows us to materialize a lot of AI applications. Though these applications still are primitive or awkward, but they fairly work. For example this Sony Aibo Robot Dog



    It can do a lot, but not "naturally" enough as a real dog does. Further: Aibo cannot fall in love with another Aibo, can it? But the computing power is already amazing for some of us. The AI-driven Aibo bases on a sum of different knowledges which must be computed and deducted so that the movements become fluent. The more powerful the CU is, the more natural (i.e. fluent) the movements will be.

    Memory: Memory is passive, but vital for AI. Working Memory (RAM or Cache) and persistent Memory (Repository or Database) are the base of every AI interpretation and execution. The reason is simple, but unaware by some people: Knowledge is BigData and big data require more memory to work with (RAM/Cache) or to stow (Repository or Database).

    Reliable Input/Output Units (RioU): like human AI is not AI without RioU. Human needs eyes (visual), ears (sound), skin (sensing), nose (smelling), etc. AI needs sensors to interface the outer world -the human world. Like human's RioU AI-RioU must be reliable and steady available. Unlike human RioU which can be "kaputt" (broken) AI-RioU must be always functioning. The Boeing 737 Max 8 fate is the proof of a bad-functioning RioU (sensor at the airplane nose).

    Working Units (WU): The RioU are controlled by WU which must be able to coordinate and to work in sync with other WU. On human a noise (ears) triggers a head movement to the source of the noise so that the eyes could visualize the cause of the sound -just for a knowledge about the sound. A complex cooperation between different WU: ears, neck muscle, eyes. The more AI-WU frictionlessly cooperate the more AI becomes natural. Aibo dog needs more synchronized cooperation between the WU in oder to behave like a real dog.

    [​IMG]
    (Source: IBM)
     
    Last edited: 8/4/19
  3. Joe

    Joe Thành viên VIP

    2. Fuzzy Logic (FL)

    What is fuzzy? Fuzzy is in lay language "unsharp", "unclear", "blurred", "vague". Fuzzy Logic is a logic that expresses an statement which is relatively near, but not exactly to the core. It's the nature of human whose expression is usually blurred, unclear and sometimes quite ambiguous. For example: "I don't feel good". The rating of "don't feel good" becomes ambiguous and unsharp, too. Such an "expression" would lead to the question that would be "why you don't feel good?", or "what's wrong?". And the answer for the question is just "a bit" more precise, but still quite ambiguous (e.g. "I drank yesterday too much", or "I got a bad night", etc.). Such an ambiguity is the logic of "free rating" and "free rating" lets no room for an alternative.

    Lotfi Zadeh, an American Mathematician, introduced Fuzzy Logic in 1920:
    Impreciseness and Vagueness are the foundation of Fuzzy Logic.

    Fuzzy Programming (FP)

    It doesn't mean that one has to code "precisely" in Zadeh's FL in order to have FL. Because FL is itself "imprecise" the fuzziness that leads to the goal is decisive. Hence Fuzzy Programming can be materialized with conventional "technique", too. As said, the goal is decisive. The way how to achieve the goal is secondary, peripheral. And that can be done the best in any OOPL such as JAVA or PYTHON or C/C++. CongdongJava is a JAVA community and so Fuzzy Programming is here in JAVA. Fuzzy Programming is useful and so some case the best to reduce the overload (response time) and the excessive use of memory (performance.)

    Let start with an example. Suppose we have to build a Web Proxy to interface Browsers and the Web. The traffic between browsers and the Web is immense -big data if you so will. The problems for a web proxy are how
    • to pass efficiently the requests of different browsers (i.e. users) to the web, and
    • to handover the replies from the web to each individual browser.
    It's clear that the classic architecture is Client-Server where the clients are the browsers and the web and the server is the Web Proxy. The server produces each thread that serves in between browser and the web. The latent time between browser and the web is crucial. This latent time is the processing time of server or the Web Proxy. The longer the processing time is, the more the latent time becomes. And the latent time is for the users no other thing than the response or reaction time of the server. A long response time is for the user usually inacceptable.

    The rule of Fuzzy Programming is simple. It's restricted itself on the knowledge that the data are bound to a known set of values (as keys or commands) and each value is unambiguous. The Browser requests base on the HTTP protocol which has a set of unique commands like GET, PUT, POST, CONNECT, etc. And the Web responses base on the HTTP protocol, too. With this knowledge we are able to implement a Web Proxy in Fuzzy Programming technique.

    An example of a Browser's HTTP Request to the Web and the Web's HTTP Response:
    PHP:
    // HTTP Request
    GET /... HTTP/1.1
    ...
    Connectionkeep-alive
    Accept
    : ...
    User-AgentMozilla/5.0 (Windows NT 10.0WOW64AppleWebKit/537.36....
     
    ...
    // HTTP Response
    HTTP/1.1 200 OK
    Date
    Fri27 May 2016 14:43:04 GMT
     
    ...
     
    But a Browser's HTTP Request to the Web Proxy is slightly different:
    PHP:
    GET /... HTTP/1.1
    ...
    Proxy-Connectionkeep-alive
    Accept
    : ...
    User-AgentMozilla/5.0 (Windows NT 10.0WOW64AppleWebKit/537.36 ...
    ...
    The prefix Proxy- is inserted in front of the line "Connection: keep-alive". This prefix is unique and one of the HTTP rules is that the main keywords must always begin at a New Line after Carriage Return ("\r\n"). Knowing that we "fuzzy-program" to intercept the Browser Request and manipulate it so that the Web believes to be connected directly with the browser (not with the Web Proxy):
    PHP:
    // Conventional Programming
       
    ByteBuffer wbuf ByteBuffer.allocateDirect(2048);
       
    int len soc.read(wbuf); // read 2KB from Browser SocketChannel
       
    ((ByteBuffer)wbuf.flip()).get(buf0len);
       
    SocketChannel webSoc SocketChannel.open(new InetSocketAddress(webHost80)):
       
    // search for "Proxy-Connection: keep-alive"
       
    String req = new String(buf0len); // 1. object more with the size of len
       
    req req.replace("Proxy-"""); // 2. and 3. object more (new req, "Proxy-")
       
    buf req.getBytes(); // 4. object more (new buf)
       
    webSoc.write(buf);
       ...
    PHP:
    // Fuzzy Programming
       
    ByteBuffer wbuf ByteBuffer.allocateDirect(2048);
       
    int len soc.read(wbuf); // read 2KB from Browser SocketChannel
       
    ((ByteBuffer)wbuf.flip()).get(buf0len);
       
    SocketChannel webSoc SocketChannel.open(new InetSocketAddress(webHost80)):
       
    // search for "Proxy-Connection: keep-alive"
       
    for (int i 15len; ++i// start after "XYZ / HTTP/1.1\r >= 15
         // Checking for "Proxy-" at position
         // -  0 for byte \n (we know \n is after \r)
         // -  1 for the letter 'P' we look for P(roxy)
         // -  5 for the letter 'y' we look for (Prox)y
         // if all 3 conditions are met we assume that the word "Proxy-" was found.
         
    if (buf[i] == '\n' && buf[i+1] == 'P' && buf[i+5] == 'y') { // pos.0,1,5
           
    System.arraycopy(bufi+7bufi+1len 6); // remove "Proxy-"
           
    len -= 6;
           break;
         }
       }
       
    webSoc.write(buf0len);
       ...
    The Conventional Programming technique looks quite simple and clear. But it produces 4 following additional objects that must be later "garbage-collected" (bad for Response Time):
    • String req
    • new String Req (due to String immutability)
    • String "Proxy-"
    • old buf (due to array immunity)
    The Fuzzy Programming technique does look a bit "confusing", but faster and almost as precise as CP whereby no additional object was needed (less Garbage Collection.)

    Fuzzy Programming Rules
    FP bases on two assumptions:
    • Assumption that a keyword (e.g. "Proxy-Connection") or a pattern is then found if n letters match the same positions of the keyword/Pattern (0, 1 and 5). Instead of Checking for the whole keyword/pattern (or part of the keyword/pattern) only n unique letters are check at n determinate positions within the keyword. Pattern Recognition bases on such a Fuzzy Programming.
    • Assumption that the Probability (the "rate of Similarity") with n samples (here: the positions) should be so reliable that a deduction could be optimally achieved (up 0.5).

    [​IMG]
    Map Reduce Algorithm (source TheGeekStuff.com)

    Google Search Engine (GSE) works with Map Reduce Algorithm or MRA which is no other thing than Fuzzy Programming. MRA calculates the so-called "Term Frequency" or TF which is the frequency (or occurrence) of a term (or an expression, e.g. "Proxy") within a document (or object) and then the Inverse Document Frequency or IDF which is the weight of TF within a heap of documents. With TF-IDF GSE is able to deduct some search results whether they are the same or just similar (e.g. how much similar?) The deduction is based on the calculation "CosineSimilarity".

    Some words about CosineSimilarity: We know in the trigonometry that cos(0°) is 1 and cos(90°) is 0. A range between [0, 1]. An angle of 90° indicates a perpendicularity. An angle of 0° signifies a parallelism of objects. Parallelism is in the geometry the similarity between two lines: the CosineSimilarity.
    (Wikipedia: CosineSimilarity)

    And AI bases on Similarness, Commonness, Approximateness and Relativeness.

    Back to our Web Proxy: with FP we can enhance our Web Proxy with more features, for example, a blacklist (of perversive ads), Cookie-Filter, a bridge to other networks (e.g. TOR network), etc. The full codes can be downloaded here (either with ZIP or with JZIP).
     
    Last edited: 12/4/19
  4. Joe

    Joe Thành viên VIP

    (cont.)
    Fuzzy Logic

    The "pure" lore of Fuzzy Logic is the work of Lotfi Zadeh. FL relies on Mathematical Probability and basses on Set Theory (ST) and for "the ease of vague understanding".

    Fuzzy Logic is more real than "computer logic" where the reasoning is always unambiguous. EITHER this (result) OR that (alternative). FL is vague and approximate and there's NO alternative. Example:
    Computer Logic (CL)
    PHP:
    if == b then c // result
    else           // alternative
    Fuzzy Logic
    PHP:
    if a similar b then c is a
    • CL is binary and crisp while FL is ambiguous and vague.
    • CL operators are absolute (==, =, etc) while FL operators are relative (similar, is, etc.)
    • CL object have two distinct states (entities) which are determinable while FL objects possess numerous states which can be vague and overlapping.
    • CL state (or data) is final while FL state (or data) is variable and imprecise.
    Because FL is the synonym of Similarity, of Vagueness and of Approximateness FL data are indeterminate, imprecise and so FL variables become ambiguous. Thus Operations with FL data and FL variables are also vague, indeterminate and ambiguous. Such an absoluteness like Equal (==), Greater (>), etc. is too precise in oder to render an "ambiguous state" of an object which is in reality very common (e.g. cold, hot, warm, etc.)

    FL Operations must be able to operate on the vague state of FL objects so that any transition between the states could be rendered. Because there's NO distinct transition between different states the ST operations (AND, OR, INVERSE, EXCLUSIVE/INCLUSIVE OR, INVERSE, etc.) can be optimally applied to FL. I don't go into details with Set-Theory operations, but hint you to see the given link above, or to Google for more if you want to do some researches on ST issues. Or THIS.

    In order to render numerous states of a FL object FL Data must be a set of various ranges. Example: FL data for temperature:
    • cold is between -10° and +10°,
    • cool is between +8° and +18°,
    • fair is between +15° and +20°,
    • warm is between +18° and +26°,
    • hot is between +24° and +34°,
    The data of Temperature can be extended, for example, to: very cold, absolute cold, very hot, boiling hot, etc. The state of each data range is floating between 0.0 to 1.0 (a double or a float, -1.0 indicates the out of range) and has a form of either a pyramid or a trapezoid as following
    [​IMG]
    The Y-Axis is the "Fuzzy Axis" and the X-Axis is the Sample-Axis. The perpendicular line that cuts the left or right line of the pyramid (or trapezoid) is the sample line. On the Sample-Axis it's the sampled value. The projection of the cutting point on Fuzzy-Axis gives the Fuzzy value. The calculation of the left and right area is called the "defuzzification". The result is then the final outcome of FL operations. Defuzzification can be computed as the Average which can be "fined-grained" or "coarse", or as the Center (or Centroid). An example of a FL operation:

    PHP:
    if temperature is cool AND AirCon is low then room is cool
    As we know, FL does not provide an alternative, hence the outcome is singular: room is cool. Nothing more. Nothing less. And the outcome (room) is also vague and ambiguous: cool between 8° and 18°. AI bases on knowledge and knowledge is big data. Data with complex and intricate ramification. Face or fingerprint or linguistic recognition is for example the most intensive R&D of today. However the results are mediocre. It isn't the inability of researchers and developers, it's the intricate ramification. And ramification can only be rendered by similarness and commonness of some certain traits. With FL we could convey some properties which mirror these features of the observing object. Face recognition appraises and measures, for example, the locations and forms of the eyes, mouth and nose and then deduct the face that has the most similar traits with a reference in an existing repository. Such a work is very time-consuming without FL.

    Machine Learning is another field that requires lots of detailed data. Big Data. If Face Recognition is extended to include animal faces more references must be established and validated (supervisor or unsupervisor). The locational details of eyes, mouth and nose must be reworked and categorized (human, animals, birds, insects). FL provides here the base for ML: dog-like (coyote, wolf, chihuahua, etc.), feline (lion, tiger, chetah, etc.) and so on.

    Some IT developers think that AI needs AI-oriented language, Front-End requires JS and PHP, etc. They are wrong. All programming languages are suited to code AI applications. The knowhow, not the language itself is the key. JAVA or C/C++ is suited for AI as well. PYTHON is an interpretative language which can directly execute (the source codes) without compiling. And that was the lure to code in PYTHON (by researchers or developers who come from other fields than IT) to see how their AI-Idea works. Hence it's simple to work with "type-less" variables (int, float, double, etc.) because their types will be recognized and adjusted by PYTHON during the interpretative phase. And that can be done with JAVA or C/C++, too, as an AI engine which interprets the AI codes and execute them. Similar to an "AI-Engine" FL does not need a special FL language, but an FL engine that interprets the codes and executes them -exactly as PYTHON does.

    Example: the following JAVA codes are the reduced codes of my FL-Engine. The excerpts show you how easily a FL-Engine is implemented in JAVA. The interpretive work may look complicated, but it isn't. It's the basic implementation technology of Interpreter. The FL result is FUZZY. Meaning: vague, imprecise. To attain the "crisp" values FL-Variable must be "defuzzified". The Defuzzification is the weighting of all set samples of FL-Variable. The calculation bases on the left and right area of each sample. If the computing is either
    • fine-grained, or
    • coarse, or
    • centroid
    FL depends on FuzzyData and how the states (data) are arranged as pyramid or trapezoid. If the arrangements are contradictory FL could deliver contradictory result, too.

    The "ancient" naming for "Machine Learning" is "Automatization". Due to limited memory and mediocre processor "ancient" automatized machine is good, but not as intelligent as the modern "Machine" which can learn. Hence: Machine Learning. As discussed in previous chapter ML is AI that bases on a big repository knowledge. And knowledge is Similarness, Commonness, Approximateness and Relativeness. Such requirements suit the best Fuzzy Logic and FuzzyData is a state which describes an entity and ranges from X to Y -not determinate, but floating.

    The given example Room.java is a simplified Machine Learning in both modes: Supervisor and Unsupervisor. With 2 parameters Room can be controlled either by a "Supervisor" or "itself" the Supervisor (Unsupervisor).
    PHP:
    import fuzzylogic.FuzzyEngine;
    // Joe Nartca (C)
    // 2 parameters
    // - 1st parameter (a double) is the outside temperature
    // - 2nd parameter (anything) is for Machine Learning.
    // 1 parameter: either a double (outside temperature) or anything (see 2nd parameter)
    // NONE: default. -10 Celsius, Supervisor
    public class Room {
      public static 
    void main(String... argsthrows Exception {
        
    double outside = -10d;
        
    boolean auto false;
        if (
    args.length 0) {
          try {
            
    outside Double.parseDouble(args[0]);
            if (
    args.length == 2auto true;
          } catch (
    Exception ex) {
            
    outside = -10d;
            
    auto true;
          }
        }
        
    // preset Room Temperature between -10 to 35 Celsius
        
    double temp outside < -10d? -10d:(outside 35)? 35:outside;
        ...
        
    FuzzyEngine eng = new FuzzyEngine();
        
    // Temperature
        
    eng.defFuzzyData("cold", -100d0d10d);
        ...
        
    eng.defFuzzyVariable("room");
        
    eng.addFDtoFV("room""cold");
        ...
        
    // AirConditioner
        
    eng.defFuzzyData("min", -10.5d, -8d, -6.5d);
        ...
        
    eng.defFuzzyVariable("AirCon");
        
    eng.addFDtoFV("AirCon""min");
        ...
        
    // temp
        
    eng.defFuzzyVariable("temp");
        
    eng.addFDtoFV("temp""cold");
        ...
        
    double rate 0.0d;
        
    // the range of values 20d...22d is the "knowledge" (known in repository)
        
    while (temp 20d || temp 22d) {
          
    eng.setFVSample("room"temp);
          if (
    auto) { // Unsupervisor Machine Learning
            
    eng.execute("if room is cold then AirCon is max");
            
    eng.execute("if room is cool then AirCon is high");
            
    eng.execute("if room is fair then AirCon is OFF");
            
    eng.execute("if room is warm then AirCon is low");
            
    eng.execute("if room is hot  then AirCon is min");
            
    rate eng.defuzzy("AirCon"0);
          }
          
    eng.setFVSample("AirCon"rate);
          
    eng.execute("if room is cold and AirCon is min then temp is cold");
          
    eng.execute("if room is cool and AirCon is min then temp is cool");
          
    eng.execute("if room is fair and AirCon is min then temp is fair");
          
    eng.execute("if room is warm and AirCon is min then temp is fair");
          
    eng.execute("if room is hot  and AirCon is min then temp is warm");
     
          
    eng.execute("if room is cold and AirCon is low then temp is cool");
          
    eng.execute("if room is cool and AirCon is low then temp is cool");
          
    eng.execute("if room is fair and AirCon is low then temp is fair");
          
    eng.execute("if room is warm and AirCon is low then temp is fair");
          
    eng.execute("if room is hot  and AirCon is low then temp is warm");
     
          
    eng.execute("if room is cold and AirCon is OFF then temp is cold");
          
    eng.execute("if room is cool and AirCon is OFF then temp is cool");
          
    eng.execute("if room is fair and AirCon is OFF then temp is fair");
          
    eng.execute("if room is warm and AirCon is OFF then temp is warm");
          
    eng.execute("if room is hot  and AirCon is OFF then temp is hot");
     
          
    eng.execute("if room is cold and AirCon is high then temp is cool");
          
    eng.execute("if room is cool and AirCon is high then temp is fair");
          
    eng.execute("if room is fair and AirCon is high then temp is warm");
          
    eng.execute("if room is warm and AirCon is high then temp is warm");
          
    eng.execute("if room is hot  and AirCon is high then temp is hot");
     
          
    eng.execute("if room is cold and AirCon is max then temp is fair");
          
    eng.execute("if room is cool and AirCon is max then temp is warm");
          
    eng.execute("if room is fair and AirCon is max then temp is warm");
          
    eng.execute("if room is warm and AirCon is max then temp is hot");
          
    eng.execute("if room is hot  and AirCon is max then temp is hot");
          
    temp eng.defuzzy("temp"0);
          if (!
    auto) { // Supervisor Machine Learning
            
    rate += temp 200.05d: -0.05d;
            
    rate  rate < -10drate 1d:rate 10drate:rate 1d;
          }
        }
        
    System.out.printf("Outside: %2.2f Celsius, Room: %2.2f Celsius, AirCon: %2.2f\n",
                          
    outsidetemprate);
        ...
      }
    }
    The following excerpt shows you how FuzzyLogic Expressions are interpreted and executed.
    PHP:
    package fuzzylogic;
    // Joe Nartca (C)
    public class FuzzyEngine {
      public 
    FuzzyEngine( ) {
        ...
      }
      public 
    void defFuzzyVariable(String name) {
        ...
      }
      
    // FD pyramid
      
    public void defFuzzyData(String namedouble leftdouble topdouble right) {
        ...
      }
      
    // FD trapezoid
      
    public void defFuzzyData(String namedouble leftdouble ltopdouble rtopdouble right) {
        ...
      }
      ...
      
    // Simplify FuzzyEngine with only 2 ST-Operations: AND and OR
      // FL Expresion: if FV1 is FD1 and/or FV2 is FD2 ... then FVr is FDr
      
    public void execute(String fuzzythrows Exception {
        
    String[] terms fuzzy.split(" ");
        
    ArrayList<StringfExp = new ArrayList<String>(terms.length);
        for (
    String s:terms) if (s.length() > 0fExp.add(s);
        if (
    fExp.size() < 8) throw new Exception("Incomplete FuzzyExpression:"+fuzzy);
        
    boolean ok falseboolean IF = true;// syntax check
        
    for (int i 0mx fExp.size(); mx; ) {
          if (!
    fExp.get(i++).equalsIgnoreCase("if") && IF)
            throw new 
    Exception("Expected \"if\" but found:"+fExp.get(i-1));
          if (!
    FVs.containsKey(fExp.get(i++)))
            throw new 
    Exception("Expected \"FV\" but found:"+fExp.get(i-1));
          if (!
    fExp.get(i++).equalsIgnoreCase("is"))
            throw new 
    Exception("Expected \"is\" but found:"+fExp.get(i-1));
          if (!
    FDs.containsKey(fExp.get(i++)))
            throw new 
    Exception("Expected \"FD\" but found:"+fExp.get(i-1));
          if ((
    i+1) >= mx) {
            if (!
    ok) throw new Exception("Invalid Syntax:"+fuzzy);
            break;
          }
          
    String op fExp.get(i);
          
    ok op.equalsIgnoreCase("then");
          if (!
    op.equalsIgnoreCase("and") && !op.equalsIgnoreCase("or") &&
              !
    ok) throw new Exception("Expected \"Operator\" but found:"+op);
          if (
    >= mx) throw new Exception("Invalid Syntax:"+fuzzy);
          IF = 
    false;
        }
        
    // execute
        
    IF = true;
        
    ok false;
        
    double result = -1d;
        for (
    int i 0mx fExp.size(); mx+= 4) {
          
    String op fExp.get(i);
          
    String fvName fExp.get(i+1);
          
    String fdName fExp.get(i+3);
          
    FuzzyVariable fv FVs.get(fvName);
          if (IF) {
            IF = 
    false;
            
    result fv.isFuzzy(fdName);
          } else if (
    op.equalsIgnoreCase("then")) {
            if (
    result != -1dfv.addPoint(fdNameresult);
            return;
          } else {
            
    double d fv.isFuzzy(fdName);
            if (
    op.equalsIgnoreCase("and")) {
              if (
    resultresult d;
            } else { 
    // it's an OR
              
    if (resultresult d;
            }
          }
        }
      }
      ...
    }
    Download the room.zip (conventional ZIP file) or room.txt (the JZIP file) and un(j)zip it. The content is:
    • FuzzyData.java
    • FuzzyVariable.java
    • FuzzyEngine.java (reduced from Joe's FuzzyEngine.java)
    • Room.java
    and if you've successfully compiled the sources and run the "Room" the Room's output is as following:
    PHP:
    C:\links\java\AI_FL>javac -g:none -./classes Room.java
     
    C
    :\links\java\AI_FL>java Room 100
    Outside
    100,00 CelsiusRoom20,40 CelsiusAirCon: -10,00
     
    C
    :\links\java\AI_FL>java Room 30
    Outside
    30,00 CelsiusRoom21,80 CelsiusAirCon: -9,50
     
    C
    :\links\java\AI_FL>java Room 50
    Outside
    50,00 CelsiusRoom20,40 CelsiusAirCon: -10,00
     
    C
    :\links\java\AI_FL>java Room -50
    Outside
    : -50,00 CelsiusRoom21,80 CelsiusAirCon7,00
     
    C
    :\links\java\AI_FL>java Room -10
    Outside
    : -10,00 CelsiusRoom21,80 CelsiusAirCon7,00
     
    C
    :\links\java\AI_FL>java Room -1
    Outside
    : -1,00 CelsiusRoom20,40 CelsiusAirCon7,00
     
    C
    :\links\java\AI_FL>java Room -1 auto
    Outside
    : -1,00 CelsiusRoom21,80 CelsiusAirCon7,95
     
    C
    :\links\java\AI_FL>
    NOTE: next article I'll show you how to implement a full-functional FuzzyEngine (syntax and scripts) which is partly shown here (FuzzyData, FuzzyVariable and simplified FuzzyEngine)
     

    Attached Files:

    Last edited: 14/4/19
  5. Joe

    Joe Thành viên VIP

    3. Repository

    "Knowledge is power!" says a proverb. And Francis Bacon said
    With Knowledge AI knows what AI's doing. And Knowledge must be kept and just-in-time available when AI needs it. Otherwise AI is not lively, but stonedead. I am talking about the AI repository. The brain (of every living species) is an unmatched repository with a lightning fast access. You see a face and know either this face is familar or unknown -less than a blink. That's the power of the brain and that makes every living species superior against any man-made robot -even the skrewdest AI robot.

    AI needs also its own Repository -just like every living species. The knowledge can be shared (as information), but not mutually owned. And Repository is a storing vault which is known in IT as the Database (DB). However DB is more or less very simple-structured compared to the structure of the brain. The Relational DB requires keys to access the data which are alas so "normalized" that the objects themselves are irrecognizable. Or Normalization in other words: ACID. And it stands for: Atomicity (in lay language: indivisible), Consistency, Isolation and Durability. For example, the term Isolation alone makes the data unrelated to their surrounding. But Knowledge always correlates with its surrounding. Two eyes, for example, are only recognizable in correlation with a face: human or animal or insect. The consequence is that RDB is very ill-suited as the DB for an AI Repository. Object is object and cannot be "atomized". A car is a complete object and as a car, nota combination of 4 wheels, hood, engine, etc. And the AI world is the same world seen by human (or AI Object-Oriented developers):

    [​IMG]
    (Source: Medium.com)

    Knowledge about Vehicle leads to different types of vehicles, the type says about the object and the object indicates its own identity (e.g. sedan, SUV or hatchback). Such an object-reasoning and identifying require a specific object-oriented DB (OODB) or OO-Repository. However commercial OODB is too generic to fit for an AI work. This link gives you an overview about 9 available OODBs. I am myself never involving with any of these OODB. It's in my belief that AI needs OODB which is specialized to each individual AI purposed application. We are not able to cover everything like our brain, hence it's more apt to tailor a specific AI OO-Repository (OOR) for a specific application. It sounds costly and wasteful, but it's in my opinion cheaper and more useful for a long term thinking. Technology evolves and is always a new base for a new technology. AI developers don't need to develop a versatile OOR (like coding with PYTHON -regardless to its interpretative slowness), but see the additional OOR work as a necessary coding work to achieve the most optimal AI application. Mother Nature has showed us that every living species has its own OOR. It isn't a costly-and-wasteful work of Mother Nature.

    OOR should be also a "home-made" OODB because a generic AI is impossible and unfeasible. Face or Finger Recognition is, for example, different to DataMining with an AI-search algorithm. Their data are specific and therefore they must be treated differently. Java (or other OOPL) Objects are linguistic object-bound and can only be processed by their own methods written in Java. And that makes sense to build or to develop an OOP-specific OOR. To do that Java Objects need only to be Serialized.

    Serialized Java Objects can be stored, fetched and processed by their own "knitted" OODB methods. The schematic of an OOR can be simple (flat file) or complex like a professional OODB (Client-Server). It all depends on the requirements and your IT ability to be capable to develop such an extra work. Coding is already a work. If one learns to code one should learn to design one's own "data container", too. The Object-Oriented Repository.

    Implementation

    The (OO)DB foundation is simple: IO activities. It's the lowest level. All DB-IO activities rely and to be dependent on the host Operating System and its subsystems (disk, USB, etc.). IO means file IO. File leads to file-system. And you have two choices: use the default file structure or invent your own.

    The first option is simple, but it does not give you much room to maneuver. You have to work with OS conventions and OS formats -with all their advantages and disadvantages. The last option requires you, as DB Designer-Implementer, to be creative and inventive. Here you can set your own conventions, your own formats and your own (file) structure. I show you hereunder the last option.

    Space saving is disk space saving. When you design an architectural structure of your DB file you need to know:

    - How can I save my data optimally? Are there different ways to compress data ? Also, what compress technique should I use ? Does the compression process take more time than a system-conventional WRITE of NON-compressed data?
    - How can I read my compressed data as fast a s possible ? Does the decompression process take more time than a system-conventional READ process of NON-compressed data ?
    - How can I optimize the compression and decompression process ?
    - How can I access the data correctly and efficiently ?

    JAVA IO and JAVA NIO both give us in plenty of possibilities to design and to implement our own DB file. First of all, we define our DB file format

    [​IMG]

    An implementation example:
    PHP:
      // java.io
      
    public void open( ) throws Exception {
        if (!(new 
    File(dbName)).exists()) { // create an empty file
          
    try (FileOutputStream fo = new FileOutputStream(dbNamefalse)) {
            
    fo.close();
          } catch (
    Exception ex) {
            throw new 
    Exception("Unable to create OODB:"+dbName);
          }
        } else try (
    FileInputStream fi = new FileInputStream(dbName)) {
          if (
    fi.available() > 0) { // DB file is NOT empty
            // load and decompress DB data (byte-wise)
            
    GZIPInputStream gi = new GZIPInputStream(fi);
            
    ByteArrayOutputStream bo = new ByteArrayOutputStream();
            
    byte[] rec = new byte[MAX_BUF]; // 64K
            //
            
    int p;
            for (
    gi.read(rec); != -1gi.read(rec)) bo.write(rec0p);
            
    rec bo.toByteArray();
            
    gi.close();
            
    bo.close();
            
    // cache keys and objects
            
    0;
            while (
    rec.length) {
              
    // compute key length
              
    int kL = (int)((rec[p++]&0xFF) << 8)+
                       (int)((
    rec[p++]&0xFF));
              
    // and object length
              
    int oL = (int)((rec[p++]&0xFF) << 24)+
                       (int)((
    rec[p++]&0xFF) << 16)+
                       (int)((
    rec[p++]&0xFF) << 8)+
                       (int)((
    rec[p++]&0xFF));
              
    // retrieve object
              
    byte[] obj = new byte[oL];
              
    String k = new String(recpkL);
              
    //
              
    keys.add(k);
              
    System.arraycopy(recp+kLobj0oL);  
              
    cache.put(kobj);
              
    += (kL+oL);
            }
          }
        } catch (
    Exception ex) {
          throw new 
    Exception("Cannot open:"+dbName);
        }
        
    closed false;
      }
    With Java FileChannel (java.nio)
    PHP:
      // java.nio
      
    public void open( ) throws Exception {
        if (!(new 
    File(dbName)).exists()) {
          ...
        } else try (
    FileChannel fc = (new FileInputStream(dbName)).getChannel()) {
          if (
    fc.size() > 0) { // DB file is NOT empty
            
    ByteArrayOutputStream bo = new ByteArrayOutputStream();
            
    ByteBuffer bb ByteBuffer.allocateDirect(MAX_BUF);
            
    // load and decompress DB data (block-wise)
            
    int p;
            
    byte[] rec = new byte[MAX_BUF];
            for (
    fc.read(bb); 0fc.read(bb)) {
              ((
    ByteBuffer)bb.flip()).get(rec0p);
              
    bo.write(rec0p);
              
    bb.clear();
            }
            
    fc.close();
            
    GZIPInputStream gi = new GZIPInputStream(new ByteArrayInputStream(bo.toByteArray()));
            
    bo.reset(); // clear ByteArrayOutputStream
            
    for (gi.read(rec); 0gi.read(rec)) bo.write(rec0p);
            
    rec bo.toByteArray();
            
    gi.close();
            
    bo.close();
            
    // cache keys and objects
            
    0;
            while (
    rec.length) {
              ....
            }
          }
        } catch (
    Exception ex) {
          throw new 
    Exception("Cannot open:"+dbName);
        }
        
    closed false;
      }
      
    The FileChannel implementation applies nio to exploit the bock-wise IO processing. It looks a bit awkward and somehow slow. Yes. It could be slower than the byte-wise implementation. But as long as the DB file is small (less than 100KB) FileChannel is at disadvantage. The situation reverses when the DB file size increases (up more than 100KB).

    Using GZIPInputStream and GZIPOutputStream we could reduce the used disk space up to 67 % (see Screenshot.)
    [​IMG]

    With this OOR structure we are able to develop lots of derivatives what I've done with the JZIP application (see ZIP or GNU-ZIP)
     
    Last edited: 21/4/19
  6. Joe

    Joe Thành viên VIP

    Before I shows some examples let's complete the self-built OO Repository and talk about Design & Implementation of Data Integrity and Data Consistency. The odb-package can be downloaded hereunder.

    In Part I we design and implement our own file system with the following entry-format (or record format):

    PHP:
    0            2              6                k           n
    +-----------+--------------+----------------+-------------+
    KeyLength ObjectLength Key/ObjectName Object      !
    +-----------+--------------+----------------+-------------+
    2 bytes KeyLengthmax64K
    4 bytes ObjectLength
    max 2GB
    k bytes Key
    /ObjectNamemax64 KB
    n bytes Object
    max2GB
    • Key or ObjectName-Length (2 bytes) indicates the size of object name. Max. 64KB or a string of 64KB
    • ObjectLength (4 bytes) is the size of the object itself in bytes. Each object can have max. 2 GB (BigData)
    • Key/ObjectName is the readable name of the following object. It must be unique.
    • Object with the size of max. 2G Bytes. An object could be a serialized JAVA object, or a byte-array which could be object of other OOPL, a content of an image/sound file, or an URL link to another object, etc.
    Note: if the byte array of an object is the content of a file (e.g. image) it's wise to use its name as the key/objectName (e.g. JoePicture.jpg). However, image/sound files are usually big it's better to keep them where they are and use their name as reference (absolute path or URL if they are on the WEB).

    Then we apply the GZIP compress technology to our dbFile in order to reduce the physical size of our dbFiles (up to 60% -see screenshot of Part I).

    Depending on the "UserRight" open() automatically create an empty DB file if there isn't a dbFile under the given dbName. The UserRight could be: ReadOnly, ReadWrite and ReadWriteDelete. ReadOnly users can only open existed dbFile. If the dbFile under the given dbName exists its content will be loaded into an ODMS cache (HashMap). Nevertheless, the caching mechanism (or algorithm) should be modified or devised if your dbFile grows too big (e.g. hundreds of objects with the size of some GB) or your DB server's scarce of RAM capacity.

    Data Integrity and Data Consistency depend on the way how we access the data, how we make sure that the data are consistent, persistent (i.e. integrity) within a multiuser environment. Before we go into details let's complete the accessing mechanism of our DB file. We set the next convention for the naming storage: for the ease of use we decide that our DB files should be grouped in a common directory -says: [current_Directory]/obb/dbName] or [current_Directory]\obb\[dbName] on WINDOWS, where dbName should be without suffix. Example: dbFile is "employee".

    PHP:
       current/working directory isC:\links\odb
       the common directory will be
    C:\links\odb\odb
       our dbFile 
    for "employee" isC:\links\odb\odb\employee
    The complete ODMS.java. This implemenation is an example showing you how the basics of an Object Data Management System is implemented. You can enhance or modified it to suit your requirement.

    Note: However, you have to mention the copyright (C) of the author Joe Nartca and Congdongjava.com as the source. The author and Congdongjava.com are NOT responsible for any loss or damage of your data. You're on your own.

    Multiuser Environment - Networking ODB-Server

    [​IMG]
    Fig.4

    In a multiuser environment users, or to be more precise applications, compete for resources. Either all resources are held exclusively by ONE user or they are shared by all users. The first option is simple to implement. User who needs a resource gets it or has to wait for it till it's release. No locking or unlocking mechanism is required. But the processing time becomes unpredictable. Due to waiting for resources some applications appear to be hung up. Such a design & implementation is obsolete.

    Sharing resources? Yes. The most optimal option. Modern design bases on the Client-Server principle: a joint server serves all clients and distributes the competing resources equally among the competitors. However, the design and implementation vary from application to application. But the principle stays unchanged, Design & Implementation could be vary from application to application. A flexible design is to build two common frameworks on each side: client and server. The frameworks are in our design & implementation the APIs (Application Programming Interfaces).

    Fig.4 depicts the scenario of communication between Clients and Server. The arrows give you a raw idea how the joint APIs work: which API works with what API. SocketChannel is the communication base between Clients and Server (block-wise). Let's follow the communicating path of our Design & Implementation. The imaginary sequence:
    • UserX sends a connection request to ODBServer (host/port).
    • ODBServer accepts UserX's request and spawns an ODBWorker which is run by JAVA ExecutorService in its StealingPool (parallel & concurrent. More: Concurrent & Parallel Programming With CPU-Affinity or read THIS)
    • ODBWorker represents its "partner UserX" on the Server side and follows its directions to deal with the specified database (or dbFiles) by commanding ODManager to handle the requests
    • ODManager invokes in its turn the in-question ODMS (one of its controlled ODMSes) to process the IO requests
    • ODMS does the IOs and hands out the result to ODManager
    • ODWorker finally picks the result (see ODBResult) and sends it back to its partner UserX.
    [​IMG]
    Fig.5 shows you how Data are cached and how they are managed. On open all data are uploaded into ODMS Cache (JAVA HashMap). All subsequent IOs happen within this Cache and in a random manner of Key-Data: also with key as the accessing index. This way (HashMap) provides the fastest IO-access time.

    Our Client-Server design bases on the following APIs:
    • ODBService. This API provides its users the possibility to design their own GUI-driven ODB Server. The invocation is simple: 3 parameters for the constructor (String config, port number and an OutputStream for the logger). If you have WEB users as clients you have to Port-Forwarding the given port. More about Port-Forwarding you'd consult your MODEM's manuals (or click HERE or Google with the keyword "Port Forwarding").
    • ODBWorker. This API is started internally by ODBService. If an user connects to ODBService it spawns an ODBWorker which is executed in StealingPool and works exclusively for this user.
    • ODBResult. The interchange of outcomes between Server and Client is wrapped in ODBResult. Again, this API is a template. You should enhance or modify this POJO to cope with your personal requirements.
    An example of ODBResult.java
    PHP:
    package odb;
    /**
     POJO Template for OODB-result after a transaction
     <br>byte array obj: reserved
     <br>boolean done, updated, existed, locked reflect the DBConnect Methods
     <br>String array Keys
     <br>String data, err, dbName
     @author Joe Nartca (C)
    */
    public class ODBResult implements java.io.Serializable {
      private static final 
    long serialVersionUID 1L;
      public 
    java.util.ArrayList<String> list;
      public 
    String keys[], err "";
      public 
    boolean bool false;
      public 
    byte[] obj;
    }
    ODBService employs ODBManager API (which manages the IOs with ODMS API) and passes it as a mutual parameter to its children (ODBWorkers). To ensure Data Integrity & Consistency, and to shield OODB from unwanted abuses Server and Client should be usually protected by Password and (unique) UserID. I include an API UserList into this odb package. However, UserList requires some protective encrypting-decrypting algorithm. The best protection is an algorithm which is known only by you yourself -nobody else! Hence you have to devise some encrypting-decrypting algorithm for yourself.

    The EnDecrypt template is as following:
    PHP:
    package odb;
    /**
    An Encrypt-Decrypt template -to be developed.
    @author Joe Nartca (C)
    */
    public class EnDecrypt {
      
    /**
        decrypt an encrypted String
        @param inp encrypted String
        @return decrypted String (null if invalid inp)
      */
     
    public static String decrypt(String inp) {
       
    // your decrypting codes
       
    return inp;
     }
      
    /**
        encrypt() a String. A dynamic algorithm using a random number as starting key
        <br>and 10-logarithm as the base to compute and to encrypt the given string
        @param inp to be encrypted String
        @return encrypted String (null if invalid inp)
      */
      
    public static String encrypt(String inp) {
        
    // your encrypting codes
        
    return inp;
      }
    }
    To ease your (temporary) work I include in the package my own EnDecrypt API (144 bits algorithm without source) so that you can immediately work with the package. Of course, you can use this API as long as you want. It's free of charge. (The link to download odb.jar will be available at the end of this series). The include userlist contains 1 user. Password: joey, userID: joe.

    The implementation of UserList API bases on a GZIP file (that is in the common odb directory). Each user entry has the following format:
    PHP:
    +------------------------------+---+-----------+
    !  
    encrypted Password:UserID   ! @ ! Privilege !
    +------------------------------+---+-----------+
    • The encrypted string (flexible length) is the concatenation of a Password string of any length and an UserID of any length, separated by a colon :))
    • @, 1 byte, is the separator between encrypted PW:uID and user's privilege.
    • Privilege, 1 byte, is one integer digit: 0 for ReadOnly, 1 for ReadWrite and 2 for ReadWriteDelete. Higher number should be reserved for further expansion (e.g. for remote User privilege)
    An example of API UserList
    PHP:
    package odb;
     
    import java.io.*;
    import java.util.ArrayList;
    import java.util.zip.GZIPInputStream;
    import java.util.zip.GZIPOutputStream;
    /**
    UserList offers 3 following functions:
    <br>1. AutoCreate a predefined gzip-file "userlist" if this file doesn't exist
    <br>2. Login Validation when login is true
    <br>3. UserList maintenance with add/delete
    <br>Privileg: Read only (0), ReadWrite (1), ReadWriteDelete (2)
    <br>Entry Format: Password:UserID@Privileg
    @author Joe Nartca (C)
    */
    public class UserList {
      
    /**
      Contructor.
      @param uFile  String, userlist file (abs.Path).
      @exception Exception thrown by JAVA (IOs, etc.)
      */
      
    public UserList(String uFilethrows Exception {
        
    this.uFile uFile;
        
    uList = new ArrayList<String>();
        if ((new 
    File(uFile)).exists()) {
          
    GZIPInputStream gin = new GZIPInputStream(new FileInputStream(uFile));
          
    ByteArrayOutputStream bao = new ByteArrayOutputStream();
          
    byte[] bb = new byte[65536];
          
    //
          
    int n;
          while ((
    gin.read(bb)) > 0bao.write(bb0n);
          
    StringBuilder buf = new StringBuilder(new String(bao.toByteArray()));
          
    gin.close();
          
    bao.close();
          
    //
          
    String[] lst buf.toString().split("\r\n");
          for (
    String s lst) {
            
    s.indexOf("@");
            
    uList.add(EnDecrypt.decrypt(s.substring(0n))+s.substring(n));
          }
        }
        
    mod false;
      }
      
    /**
      isUser
      @param pw  String, User Password
      @param uid String, User Login ID
      @return boolean false if user is unknown
      */
      
    public boolean isUser(String pwString uid) {
        for (
    String u uList) if (u.indexOf(pw+":"+uid+"@") == 0) return true;
        return 
    false;
      }
      
    /**
      getUserPrivilege()
      @param pw  String, User Password
      @param uid String, User Login ID
      @return int 0: readOnly, 1: ReadWrite, -1 if user is unknown
      */
      
    public int getUserPrivilege(String pwString uid) {
        if (
    isValid(pwuid) && isUser(pwuid)) {
          for (
    String u uList) if (u.indexOf(pw+":"+uid+"@") == 0) {
            return 
    u.charAt(u.length()-1) & 0x0F;
          }
        }
        return -
    1;
      }
      
    /**
      add new User into userlist
      @param pw  String, User Password
      @param uid String, User Login ID
      @param right int, 0: only Read, 1: ReadWrite, 2: ReadWriteDelete
      @return boolean true: user was added
      */
      
    public boolean addUser(String pwString uidint right) {
        if (!
    isValid(pwuid) || isUser(pwuid)) return false;
        if (
    right 1right 2;
        else if (
    right 0)  right 0;
        
    uList.add(pw+":"+uid+"@"+right);
        
    mod true;
        return 
    true;
      }
      
    /**
      update UserID or Password in userlist
      @param pwOld  String, User Password
      @param uidOld String, User Login ID
      @param pwNew  String, User Password
      @param uidNew String, User Login ID
      @return boolean false if invalid old pw and/or uid
      */
      
    public boolean updateUser(String pwOldString uidOld,
                                
    String pwNewString uidNew) {
        if (!
    isValid(pwOlduidOld) || !isValid(pwNewuidNew) ||
            !
    isUser(pwOlduidOld) || isUser(pwNew,uidNew)) return false;
     
        for (
    String u uList) if (u.indexOf(pwOld+":"+uidOld+"@") == 0) {
          
    uList.set(uList.indexOf(u), pwNew+":"+uidNew+u.substring(u.indexOf("@")));
          
    mod true;
          return 
    true;
        }
        return 
    false;
      }
      
    /**
      update UserPrivilege in userlist
      @param pw  String, User Password
      @param uid String, User Login ID
      @param newPriv  int, privilege. 0: Read, 1: ReadWrite, 2: ReadWriteDelete
      @return boolean false if invalid pw and/or uid
      */
      
    public boolean updateUser(String pwString uidint newPriv) {
        if (
    isValid(pwuid)  && isUser(pwuid)) {
          if (
    newPriv 1newPriv 2;
          else if (
    newPriv 0newPriv 0;
          for (
    String u uList) if (u.indexOf(pw+":"+uid+"@") == 0) {
            
    uList.set(uList.indexOf(u), pw+":"+uid+"@"+newPriv);
            
    mod true;
            return 
    true;
          }
        }
        return 
    false;
      }
      
    /**
      update UserID or Password in userlist
      @param pwOld  String, User Password
      @param uidOld String, User Login ID
      @param pwNew  String, User Password
      @param uidNew String, User Login ID
      @param newPriv  int, privilege. 0: Read, 1: ReadWrite, 2: ReadWriteDelete
      @return boolean false if invalid old pw and/or uid
      */
      
    public boolean updateUser(String pwOldString uidOld,
                                
    String pwNewString uidNewint newPriv) {
        if (!
    isValid(pwOlduidOld) || !isValid(pwNewuidNew) ||
            !
    isUser(pwOlduidOld) || isUser(pwNew,uidNew)) return false;
     
        if (
    newPriv 1newPriv 2;
        else if (
    newPriv 0newPriv 0;
        for (
    String u uList) if (u.indexOf(pwOld+":"+uidOld+"@") == 0) {
          
    uList.set(uList.indexOf(u), pwNew+":"+uidNew+"@"+newPriv);
          
    mod true;
          return 
    true;
        }
        return 
    false;
      }
      
    /**
      delete User
      @param pw  String, User Password
      @param uid String, User Login ID
      @return boolean false: unknown uid/pw
      */
      
    public boolean deleteUser(String pwString uid) {
        if (
    isValid(pwuid) && isUser(pwuid)) {
          for (
    String u uList) if (u.indexOf(pw+":"+uid+"@") == 0) {
            
    uList.remove(u);
            
    mod true;
            return 
    true;
          }
        }
        return 
    false;
      }
      
    /**
      getUserList a list of all Users
      @return String array contains all users as "pw:uid@privilege"
      <br>where [privilege] = 0 Read only, 1 ReadWrite, 2: ReadWriteDelete
      */
      
    public ArrayList<StringgetUserList( ) {
        return 
    uList;
      }
      
    /**
      Save userlist
      @exception Exception thrown by JAVA (IO, etc.)
      */
      
    public void save( ) throws Exception {
        if (
    mod) {
          
    GZIPOutputStream gzip = new GZIPOutputStream(new FileOutputStream(uFile));
          
    StringBuilder sb = new StringBuilder(uList.size());
          for (
    String s uList) {
            
    int n s.indexOf("@"); // EnDecrypt.encrypt(pw:uid) only
            
    sb.append(EnDecrypt.encrypt(s.substring(0n))+s.substring(n)+"\r\n");
          }
          
    gzip.write(sb.toString().getBytes());
          
    gzip.flush( );
          
    gzip.close( );
        }
      }
      
    //
      
    private boolean isValid(String pwString uid) {
        if (
    pw == null || uid == null || pw.length() == || uid.length() == 0) return false;
        return 
    true;
      }
      
    //--------private area
      
    private boolean mod;
      private 
    String uFile;
      private 
    ArrayList<StringuList;
    }
    Hereunder is a simple ODBServer using the API ODBService. Firstly we have to create an ODB configuration: the config.txt
    PHP:
    #
    # this is an example
    # ODB_PATH is the reserved Keyword
    #
    ODB_PATH=C:/links/odb/odb
    #
    PHP:
    import odb.*;
    // Joe Nartca
    public class ODBServer  {
      
    /**
      @param port int, Server's port
      @exception Exception thrown by JAVA
      */
      
    public ODBServer(String configint portthrows Exception {
        
    System.out.println("CRTL-C to quit");
        
    // instantiate ODBService using System.out as logger
        
    ODBService bs = new ODBService(configportSystem.out);
        
    System.out.println(".....ODB Server is running...");
      }
      
    // Localhost, port 9999 (default) or your own port number
      
    public static void main(String... athrows Exception {
        new 
    ODBServer(a.length 0a[0]:"config.txt"9999);
      }
    }
    The infinite loop
    PHP:
    while(runningpool.execute(new ODBWorker(dbSvr.accept(), odMgr));
    indicates that a PoolThread ODBWorker will be spawned directly after the waiting server (dbSvr) received an incoming Client request. The spawned thread represents the connected client to work with ODBManager. A Client can only access ODBServer (or ODBService) using ODBConnect (Client API -see Part IV). To do that it has to send a connection request with
    • the dbName it wants to connect to
    • password
    • userID
    The password and userID will be encrypted before they are sent to ODB Server. If the authentication failed ODBWorker sends an error message to its "client partner" and waits for the next "retry".

    Note: As usual, if you modify or enhance the sources you have to mention the copyright (C) of the author Joe Nartca and Congdongjava.com as the source. The author and Congdongjava.com are NOT responsible for any loss or damage of your data. You're on your own.
     

    Attached Files:

    Last edited: 24/4/19
  7. Joe

    Joe Thành viên VIP

    Some Example

    For FuzzyLogic I'll show you more on the next posting "How to implement a full-functional FuzzyEngine (syntax and scripts) and How to integrate FL into and with JAVA".

    FuzzyLogic Language (FFL) Overview
    PHP:
              FuzzyLogic KeywordsOperatotsSyntax and Operations
     
    FuzzyVariables 
    and FuzzyData must be registered before FuzzyExpressions or
    FuzzyScript can be executed.
     
    The keyword 'this' refers to JavaObject which is the Container of FuzzyEngine
    (i.ewhere FE is instantiated). Registered and Unregistered can be referred by
    FuzzyEngine
    . if Unregistered JavaObject is referredbut all the Object's
    contents (i.e. objects, primitives) are in their initial states (or values) and
    they could be null or 0.
     
    Registered JavaObjects must be instantiated, before they are registered.
    Otherwise Exception will arise when they are referred.
     
    Reference to JavaVariables (or fields): JavaObjectName:JavaVariableName, or
                                            this:JavaVariableName
    Reference to JavaMethods: JavaObjectName.JavaMethodName(Parm1, ...) or
                              this.JavaMethodName(Parm1, ...)
     
    ------------------Register FuzzyData and FuzzyVariable--------------------------
    [FDname:left, top, right]
    [FDname:left, ltop, rtop, right]
    {FVname:FDname, FDname, ..., FDname}
    --------------------------Buil-in Functions-------------------------------------
    clear FV          // clear all FV-FuzzyData of FV
    reset FV          // FV and all its FDs
    --------------------------Operations--------------------------------------------
    FV = val          // FV sample = val
    FV + val          // + FV sample with val
    FV - val          // - FV sample from val
    FV * val          // * FV sample by val
    FV / val          // / FV sample by val
    ---------------Operation with JavaVariable--------------------------------------
    FV = MyApp:temp   // set FV sample with JavaVariable temp of JavaClass MyApp
    MyApp:temp = val  // set JavaVariable to val
    MyApp:temp + val  // add JavaVariable with val
    MyApp:temp - val  // sub JavaVariable from val
    MyApp:temp * val  // mul JavaVariable by val
    MyApp:temp / val  // div JavaVariable by val
    --------------------------Statements--------------------------------------------
    if FV is FD then FVr is FDr     // assign valuated result to FDr of FVr
    if (FV is FD) then FVr is FDr
    if FV1 is FD1 or FV2 is FD2 then FV3 is FD3
    if (FV1 is FD1) or (FV2 is FD2) then FV3 is FD3
    if (FV1 is FD1 or FV2 is FD_2) and FV3 is FD3 then FV4 is FD4
    if ((FV1 is FD1 or FV2 is FD_2) and FV3 is FD3) then FV4 is FD4
    --------------------------Statements--------------------------------------------
    if FV some FD ...             // fuzzySample**2 of FD
    if FV very FD ...             // SQRT(fuzzySample) of FD
    if FV not  FD ...             // (1-fuzzySample) of FD
    if ... then FVr some FDr      // assign result**2 to FDr of FVr
    if ... then FVr very FDr      // assign SQRT(result) to FDr of FVr
    if ... then FVr not  FDr      // assign 1-result to FDr of FVr
    -----------------------Operators and Syntax-------------------------------------
    // starts a comment and ends at end of the Line
    + - * / = : .
     
    ---------------------------Keywords---------------------------------------------
    this
    reset
    clear
    some
    very
    not
    and
    or
    if
    then
    is
    --------------------------------------------------------------------------------
    and an example written in FFL script
    PHP:
    // Temperature states
    [cold: -20d, -5d5d]
    [
    cool2d10d20d]
    [
    fair19d20d22d]
    [
    warm21d23d25d]
    [
    hot:  24d30d50d]
    // AirCon regulator states
    [min: -10d, -6d, -3d]
    [
    low: -4d, -2d0]
    [
    OFF: -0.5d00.5d]
    [
    high0d2d4d]
    [
    max:  3d6d10d]
    // FuzzyVariables
    {AirConminlowOFFhighmax }
    {
    roomcoldcoolfairwarmhot }
    {
    tempcoldcoolfairwarmhot }
    // set Samples to Java Global Variables
    clear AirCon
    room 
    AHouse:temp
    if room is cold then AirCon is max
    if room is cool then AirCon is high
    if room is fair then AirCon is OFF
    if room is warm then AirCon is low
    if room is hot  then AirCon is min
    // set AirCon with defuzzied FV
    AHouse.print("Time to run")
    AHouse.print("Room:"room)
    AirCon AHouse.adjust(AirCon)
    //AirCon = AirCon
    // FuzzyExpressions
    if room is cold and AirCon is min then temp is cold
    if room is cool and AirCon is min then temp is cold
    if room is fair and AirCon is min then temp is cold
    if room is warm and AirCon is min then temp is cool
    if room is hot  and AirCon is min then temp is fair
      
    if room is cold and AirCon is low then temp is cold
    if room is cool and AirCon is low then temp is cold
    if room is fair and AirCon is low then temp is cool
    if room is warm and AirCon is low then temp is fair
    if room is hot  and AirCon is low then temp is warm
      
    if room is cold and AirCon is OFF then temp is cold
    if room is cool and AirCon is OFF then temp is cool
    if room is fair and AirCon is OFF then temp is fair
    if room is warm and AirCon is OFF then temp is warm
    if room is hot  and AirCon is OFF then temp is hot
      
    if room is cold and AirCon is high then temp is cool
    if room is cool and AirCon is high then temp is fair
    if room is fair and AirCon is high then temp is warm
    if room is warm and AirCon is high then temp is hot
    if room is hot  and AirCon is high then temp is hot
      
    if room is cold and AirCon is max then temp is fair
    if room is cool and AirCon is max then temp is warm
    if room is fair and AirCon is max then temp is hot
    if room is warm and AirCon is max then temp is hot
    if room is hot  and AirCon is max then temp is hot
    // end of FuzzyExpression
    AHouse.print("End of Fuzzy. RoomTemp:"temp)
    AHouse.print()
    AHouse:temp temp  // set Java temp with FV temp
    and the Java
    PHP:
    import fuzzylogic.FuzzyEngine;
    public class 
    AHouse {
      private static 
    double tempstepoutside;
      
    //
      
    private void print( ) {
        
    System.out.printf("Invoke by FuzzyScript: Outside %2.2f Celsius, Room %2.2f Celsius, AirCon %2.2f\n",
                          
    outsidetempstep);
      }
      private 
    void print(String msg) {
        
    System.out.println("Invoke by FuzzyScript: "+msg);
      }
      private 
    void print(String msgdouble d) {
        
    temp d// set the new temperature
        
    System.out.println("Invoke by FuzzyScript: "+msg+d);
      }
      private static 
    double adjust(double d) {
        
    double x temp 5d1d:temp 20d?0.5d:temp 25d? -0.5:-1d;
        
    step x;
        
    System.out.printf("Invoke by FuzzyScript: Defuzzy step:%2.2f -> new step:%2.2f by temp:%2.2f\n"dsteptemp);
        return 
    step;
      }
      public static 
    void main(String... argsthrows Exception {
        
    String script "Ahouse.txt";
        
    outside = -10d;
        if (
    args.length 0) {
          try {
            
    outside Double.parseDouble(args[0]);
            if (
    args.length 1script args[1];
          } catch (
    Exception ex) {
            
    script args[0];
            
    outside = -10d;
          }
        } else {
          
    java.util.Random ran = new java.util.Random();
          
    step = (double)ran.nextInt(10);
          
    outside = (double)ran.nextInt(40);
        }
        
    step 0d;
        
    // preset Room Temperature between -10 to 35 Celsius
        
    temp outside < -10d? -10d:(outside 32)? 32:outside;
        
    // FuzzyLogic..............................
        
    FuzzyEngine eng = new FuzzyEngine(this,"AHouse");
        
    // registered java Variable step of this class
        
    eng.regJavaVariable("AHouse:temp"temp);
        
    eng.regJavaVariable("AHouse:step"step);
        
    // the values between 20d and 22d are the so-called LEARNED values
        
    while (temp 20d || temp 21d) {
          
    eng.execute(script);
        }
      }
    }
    and the result is as following:
    PHP:
    C:\JFX\Fuzzy\Test>java AHouse 1 ahouse.txt
    Invoke by FuzzyScript
    Time to run
    Invoke by FuzzyScript
    Room:1.0
    Invoke by FuzzyScript
    Defuzzy step:6,20 -> new step:7,20 by temp:1,00
    Invoke by FuzzyScript
    End of FuzzyRoomTemp:20.6
    Invoke by FuzzyScript
    Outside 1,00 CelsiusRoom 20,60 CelsiusAirCon 7,20
     
    C
    :\JFX\Fuzzy\Test>
    The given ZIP file contains 5 files showing you how Map Reduced Algorithm works in "Data Mining" in a "big" directory. It's written in SWING and self-explained. The "MRA Terms" are the the searching patterns. For example: to search in a directory files which contain the string "FuzzyLogic" the MRA Term is the word "FuzzyLogic".

    If anyone of you, mon ami Nancru and all CDJ members, can post your question(s) here and I'll try my best to give you a satisfactory answer. Up 12. Mai 2019 - end Juni 2019 I'll be absent here (on vacation in north Vietnam and in south China).
     

    Attached Files:

    Last edited: 25/4/19

Chia sẻ trang này

Loading...