Artificial Intelligence And Fuzzy Logic (ai + Fl)

Joe

Thành viên VIP
21/1/13
2,885
1,288
113
Artificial Intelligence and Fuzzy Logic

1. Artificial Intelligence (AI)
2. Fuzzy Logic (FL)
3. Repository
4. Some Examples
 

Joe

Thành viên VIP
21/1/13
2,885
1,288
113
1. Artificial Intelligence (AI)

Artificial Intelligence
AI is in plain English: man-made Intelligence. Exactly as in Vietnamese: Trí Tuê Nhân tao. The word Intelligence is here in English a bit imprecise. More precise it is in Vietnamese "Trí Tuê". And "Trí Tuê" is "Knowledge". Without knowledge there's no intelligence. The right use of knowledge and intelligence is then the Wisdom.

What is intelligence? What is knowledge? The intelligence bases on knowledge. That's the matter of fact. The more knowledge accumulates, the more intelligence is increased. Human and animal as well gain knowledge by observation and deduction. A wrong deduction leads to memorized experience. And experience is the base of knowledge. Example, if we touched an unknown insect (observation) and were stung by it, and it hurt immensely (experience). From this unpleasant experience we deduct a knowledge that this insect is dangerous and we won't touch such an insect again. With this new knowledge we were able to share or to warn the others by telling them about the dangerousness of the similar insects ahead (intelligence).

Knowledge is also an accumulative process of
  1. Observation leads to inquisitiveness.
  2. Inquisitiveness leads to experimenting.
  3. Experimenting leads to experiencing.
  4. Experiencing leads to learning.
  5. Learning leads to acknowledging.
  6. Acknowledging leads to knowledge
  7. Knowledge leads to intelligence.
Intelligence is individual and personal. Even a pair of monozygotic twins doesn't share the same intelligence. It's the matter of fact that people whose knowledge could be the same draw very different conclusions. Therefore knowledge is an individual and personal deductive process. A process of 4-Nesses. The S.C.A.R.
  • Similarness, not Equalness,
  • Commonness, not Exactness,
  • Approximateness, not Resoluteness.
  • Relativeness, not Absoluteness
And intelligence bases on Knowledge. The consequence is: "natural" intelligence bases on a deductive process, not on a designated process which always produces the same equality, the same exactness, the same absoluteness and the same resoluteness. Man-made intelligence is the reflection or the mirroring of natural intelligence: AI or Artificial Intelligence is also an active process.

We have learned that a deductive process is individual and personal. AI is also individual and personal, too. A process of S.C.A.R.. Similarness, Commonness, and Approximateness Relativeness. Hence it's impossible and possibly wrong if we try to create an AI system that can be multiplied, and they all produce the very same results. The laws of S.C.A.R. are broken and we get a dumb (but somehow "shrewd") system. Not a real Artificial Intelligence system, but a passive AI System.

AI Structure
The on-the-IT-market available AI libraries such as Tensorflow, Matlab, etc. are therefore considered as the "AI automation tools" for a computerized, passive AI System. Because of the same "library" the computerized, passive AI System always produce the same decision under the same conditions. And that is against the law of individualism and personalism. If we want to build an active AI system we have to create an individual-personal system that could make decisions on its own knowledge, not on the knowledge that bases of a common-multiple-user-library. Otherwise we just have a "shrewd automat" that just follow and do exactly what the library dictates. The crash of Boeing 737 Max 8 in Indonesia (Lion Air) and in Kenia (Ethiopian Airline) is the result of a shrewd, but not intelligent automat. Boeing AI works with a sensor which delivers the needed data for a maneuvering decision. It works so far with the Equalness, Exactness, Absoluteness and Resoluteness that there's no individual and personal room for an own decision. And the results were fatal: the same crash under the same conditions by the same delivered data.

What's next? Whatsoever AI is the reflection of human intelligence. And human brain is individual and centralized if you so will. Knowledge is different from irritation or stimulation. Irritation always triggers an according unconscious reaction. For example, if our foot hits an obstacle and we fall forwards we automatically push our arms with spreading hand forwards to cushion the (heavy) fall. Such a reaction requires a quick processing of data (fall forwards) and locally (arms, hands). Therefore our spinal cord is "stretched" along the back. On the other hand, knowledge, as we already know above, bases on conscious "calculation". And calculation requires power. Our brain is therefore the most powerful Central Computing Unit -the CPU (or Processor)- with abundant "built-in cache and memory. The CPU is itself a network of neurons (Neuron Network) or a network of processors if you so will.

We human start to mimic our brain and build a "networked" computer. INTEL® CORE™ i9-9900K PROCESSOR is for example a network of 8 cores (each core is a 32-bits processor). But the computing power of our brain is still unmatched. Even with the most futuristic Quantum Computer. An "autistic child" can deliver a precise result of the most complex equation faster than any known supercomputer. With our powerful computing might we are able to store, to access and to process data instantly. The outcome is a decision, an action or whatsoever which is always the most actual. It's the deduction of knowledge. Knowledge must be existed (stored), available (accessible) and materializable (deductible).

The most basic condition for AI is also the deductibility of facts (data) which are usually presentable in mathematical or physical descriptions. For example: A circle consist of uncountable number of referent points that have the same distance to an anchor point. A pentagon is a circle with 5 referent points, a Hexagon is a circle with 6 referent points, etc. The math. description of a circle is the number referent points and their distance to an anchor point (called the radius). The knowledge of a circle is the known referent points and the radius. The decision on a shape whether it is a circle or not bases on this math. knowledge. If an AI system needs to decide on a shape with 5 referent points that share the same distance to an anchor point whether this shape a circle or not it depends on the deduction of S.C.A.R. -Similarness, Commonness, Approximateness and Relativeness. A computerized passive AI System will say that the shape is NOT a circle, but a pentagon or a hexagon. And that is the fundamental difference between active AI and passive AI (computerized machine or shrewd automat).

The S.C.A.R. laws allow dynamically to make a decision that bases on the most actual conditions. The dynamic is flexible and different from case to case.
  • Similarness: A pentagon or a hexagon resembles a circle drawn by a child. For an active AI system it's sufficient enough to deduct that the drawn shape is a circle. A passive AI system rejects such a deduction that simply bases on a resemblance.
  • Commonness: Because an infinite number of referent points is unachievable unknown shapes are recognized by their commonness. A pentagon or a hexagon share commonness: the distance of all referent points to the anchor point is the same.
  • Approximateness: the resemblance is more visible and more reliable if more referent points are sampled. Theoretically 2 referent points can be deducted as a circle with the anchor point in the middle of the line between the 2 referent points. Approximateness is crucial for a crucial decision (Boeing 737 Max 8).
  • Relativeness: the flexibility is the base of every decision. Any restriction could lead to rigidity and limits any possibility to construct a final decision in case of decision deadlock.

The AI Unsolvable
Man-made Intelligence is NOT natural Intelligence. AI bases on the Knowledge which is derived from human knowledge. Natural Intelligence is the accumulation of evolutionary knowledge. And this accumulation is invisible and inaccessible by human. It's a natural process whose evolutionary knowledge is stored over millions of years in the DNA strand.
  • Emotion is individual and personal. The rate of emotion cannot be scaled or weighted by any available mean. Every rating is always perfunctory. Never perfect. Hence AI cannot rate the feeling or gauges the emotion. The try of Volvo to monitor the face of the driver at driving and to deduct a conclusion of his moody state is deceptive because the intention of the driver to cheat is invisible for the monitoring cameras.
  • Extrapolatability is individual and personal, too. The seriousness of a sentence depends, for example, on the pitch of the speaker or the way how the speaker said. A loud you can be either a friendly warning from a danger or an unmistakeable contempt.
  • Interpretability is invisible and ambiguous. A polite rejection is usually very friendly, but uttermost unmistakable. In some cases a friendly YES is an unbridgeable, polite NO. Therefore AI-driven translator is always mediocre, never perfect (like human) and in many cases the translation turns out a nonsense.
  • Rationality and Irrationality are uttermost human. Decision depends on his temper or on some unknown reason (e.g. sympathy or apathy or hostility) and it can be very different on the very same matter. The prejudice is, for example, the most irrational decision of human. Why Vietnamese hate the Chinese so much ? Because China invaded and colonized Vietnam. Vietnam was invaded and colonized by France, too. But why Vietnamese don't hate the French as much as the Chinese? Or: why Chinese deeply hate the Japanese? Because Japan invaded and defeated China. China was invaded and defeated by Great Britain, too. But why Chinese don't hate the Brits as deep as the Japanese? It's the Prejudice. Prejudice cannot be mathematicalized or physicalized. And AI bases on Mathematics or Physics.
  • Imaginability is the most complex ability of human. The ability to see the unseen and to materialize it. For example, we imagine how "Demons", "Ghosts", etc. look like and then we sculpt them according to our imagination. It's impossible to formulate an imagination in any math. formula or to render it in any physical form. Religions are the most visible, the oldest existence of human imagination.
  • Fantasizability is also a very complex for AI-izing. Fantasy is the work of human brain which creates images or actions from nonentity. Dream or Fantasy is so individual and so personal that, for example, any description about it triggers the hearers their own dream or fantasy. Their individual and personal dreams or fantasies.
  • Learnability is the most unique ability of human (and animal). If a child got hurt by fire it quickly learns that fire is dangerous. Not the form of the fire that hurt it, but any form or appearance of fire. It's the learnability that allows human to extrapolate, to conclude and to deduct the impact.
The complexity of human Intelligence is so tremendous that a Humanoid (man-made human-like robot) can and will never match human being. Can you imagine that a humanoid falls in love with another humanoid? I can't.

AI Implementation

AI Implementation is difficult. Very very difficult. Human knowledge is an accumulation of millions of evolutionary years of knowledge which is in most circumstances impossible to reflect or to describe in mathematics or in physics. Imagination or irrationality is, for example, the product of evolutionary knowledge, not a product of Mathematics or Physics. And without Mathematics or Physics AI becomes a coded sequence of dumb actions.

Nevertheless there are a lot of human knowledge which can be solved by mathematics or physics. The Dijkstra algorithm is, for example, the search for the shortest path between two locations (or nodes) among numerous possibilities (locations). In reality we usually cut short our way in order to be faster and to lessen our effort (from walking). Dijkstra's Algorithm calculates the distances and decides which distances are the shortest so that the distance between the Begin and the Goal is the smallest sum of connected distances. This example demonstrates how Mathematics is applied to solve a human knowledge: cut short the (walking) distance. Nothing is fancy in Dijkstra math. algorithm. Only addition and comparison are involved. But the sequence of calculation-Comparison is decisive. It's the algorithm. AI is a sum of implemented algorithms. Our networked neurons (brain) possesses an unknown (or unlimited) number of sleeping algorithms which are waiting for us to be detected and to be woken up -like the Dijkstra Algorithm. The detecting base (of algorithm) is the 4-Nesses or S.C.A.R..

The weighting of one of the S.C.A.R. determines how "specific" AI implementation should be.
  • Pattern Recognizing sets its weight on Similarness. Face or Fingerprint Recognition is the most typical AI-application which works intensively with physical resemblances and mathematical similarities.
  • Machine Learning works on the base of Commonness and Approximateness. ML could be either Supervisory or Unsupervisory Learning. A supervisory AI-Learning won't make the final deduction, but rather gathers the conditions that support the deduction which is then decided by the supervisor (i.e. user). An opposite is the Unsupervisory AI-Learning wherein it relies on its learned repository knowledge and makes the deduction from the learned knowledge by itself -without any intervention of its supervisor. The bad fate of Boeing 737 Max 8 could be the problem of self-deduction made by the passive Unsupervisory ML.
  • Gaming is the most widespread AI application. Gaming ranges from Chess Computer to (Car) Navigation System. The Relativeness decides more about the deduction than the Commonness or Approximateness or Similarness. Alpha-Beta Pruning Search or MiniMax-Search, for example in TicTacToe, deducts the best move (node) from all the available moves. Or MRA (Map Reduce Algorithm) is used by Google to search for the most relatively resemblant pattern within objects out of a heap of objects.
  • Warfare and Weapons Technology are the "most lethal" AI application. Herein all S.C.A.R. are applied. Kamikaze drones, smart torpedoes, intelligent missiles and so on are the last (and the stupidest) AI research and development. Unsupervisory AI missile finds its way to the goal based on its own repository knowledge and data fed by satellites and makes its own decision where to "seppuku" in order to create the most destructive result.
However AI implementation requires a deep understanding of mathematics and a good imagination to visualize the (math) knowledge. The Cut-Short algorithm bases on two simple mathematical formulas: addition and comparison. Dijkstra's feat was the visualization of the Cut-Short knowledge with the 2 simple formulas. Simplicity is the key. Complexity is the dead end.



AI Architecture

AI Architecture is similar to our brain (as a single unit) which relies on
  • Powerful computing unit to execute complex mathematical paths (e.g. Dijkstra or MiniMax paths),
  • Plenty of memory to reposit the "gained" experiences (i.e. knowledge),
  • Reliable Input/Output units (as sensors like human eyes, ears, skin (for feeling), etc.),
  • Cross-Disciplinary working units to combine and to deduct an event in computable or representable form.
Computing Unit (CU): Contemporary technology allows us to materialize a lot of AI applications. Though these applications still are primitive or awkward, but they fairly work. For example this Sony Aibo Robot Dog


It can do a lot, but not "naturally" enough as a real dog does. Further: Aibo cannot fall in love with another Aibo, can it? But the computing power is already amazing for some of us. The AI-driven Aibo bases on a sum of different knowledges which must be computed and deducted so that the movements become fluent. The more powerful the CU is, the more natural (i.e. fluent) the movements will be.

Memory: Memory is passive, but vital for AI. Working Memory (RAM or Cache) and persistent Memory (Repository or Database) are the base of every AI interpretation and execution. The reason is simple, but unaware by some people: Knowledge is BigData and big data require more memory to work with (RAM/Cache) or to stow (Repository or Database).

Reliable Input/Output Units (RioU): like human AI is not AI without RioU. Human needs eyes (visual), ears (sound), skin (sensing), nose (smelling), etc. AI needs sensors to interface the outer world -the human world. Like human's RioU AI-RioU must be reliable and steady available. Unlike human RioU which can be "kaputt" (broken) AI-RioU must be always functioning. The Boeing 737 Max 8 fate is the proof of a bad-functioning RioU (sensor at the airplane nose).

Working Units (WU): The RioU are controlled by WU which must be able to coordinate and to work in sync with other WU. On human a noise (ears) triggers a head movement to the source of the noise so that the eyes could visualize the cause of the sound -just for a knowledge about the sound. A complex cooperation between different WU: ears, neck muscle, eyes. The more AI-WU frictionlessly cooperate the more AI becomes natural. Aibo dog needs more synchronized cooperation between the WU in oder to behave like a real dog.


(Source: IBM)
 
Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
2,885
1,288
113
2. Fuzzy Logic (FL)

What is fuzzy? Fuzzy is in lay language "unsharp", "unclear", "blurred", "vague". Fuzzy Logic is a logic that expresses an statement which is relatively near, but not exactly to the core. It's the nature of human whose expression is usually blurred, unclear and sometimes quite ambiguous. For example: "I don't feel good". The rating of "don't feel good" becomes ambiguous and unsharp, too. Such an "expression" would lead to the question that would be "why you don't feel good?", or "what's wrong?". And the answer for the question is just "a bit" more precise, but still quite ambiguous (e.g. "I drank yesterday too much", or "I got a bad night", etc.). Such an ambiguity is the logic of "free rating" and "free rating" lets no room for an alternative.

Lotfi Zadeh, an American Mathematician, introduced Fuzzy Logic in 1920:
It is based on the observation that people make decisions based on imprecise and non-numerical information, fuzzy models or sets are mathematical means of representing vagueness and imprecise information, hence the term fuzzy.
Impreciseness and Vagueness are the foundation of Fuzzy Logic.

Fuzzy Programming (FP)

It doesn't mean that one has to code "precisely" in Zadeh's FL in order to have FL. Because FL is itself "imprecise" the fuzziness that leads to the goal is decisive. Hence Fuzzy Programming can be materialized with conventional "technique", too. As said, the goal is decisive. The way how to achieve the goal is secondary, peripheral. And that can be done the best in any OOPL such as JAVA or PYTHON or C/C++. CongdongJava is a JAVA community and so Fuzzy Programming is here in JAVA. Fuzzy Programming is useful and so some case the best to reduce the overload (response time) and the excessive use of memory (performance.)

Let start with an example. Suppose we have to build a Web Proxy to interface Browsers and the Web. The traffic between browsers and the Web is immense -big data if you so will. The problems for a web proxy are how
  • to pass efficiently the requests of different browsers (i.e. users) to the web, and
  • to handover the replies from the web to each individual browser.
It's clear that the classic architecture is Client-Server where the clients are the browsers and the web and the server is the Web Proxy. The server produces each thread that serves in between browser and the web. The latent time between browser and the web is crucial. This latent time is the processing time of server or the Web Proxy. The longer the processing time is, the more the latent time becomes. And the latent time is for the users no other thing than the response or reaction time of the server. A long response time is for the user usually inacceptable.

The rule of Fuzzy Programming is simple. It's restricted itself on the knowledge that the data are bound to a known set of values (as keys or commands) and each value is unambiguous. The Browser requests base on the HTTP protocol which has a set of unique commands like GET, PUT, POST, CONNECT, etc. And the Web responses base on the HTTP protocol, too. With this knowledge we are able to implement a Web Proxy in Fuzzy Programming technique.

An example of a Browser's HTTP Request to the Web and the Web's HTTP Response:
PHP:
// HTTP Request
GET /... HTTP/1.1
...
Connection: keep-alive
Accept: ...
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36....
 
...
// HTTP Response
HTTP/1.1 200 OK
Date: Fri, 27 May 2016 14:43:04 GMT
 
...
But a Browser's HTTP Request to the Web Proxy is slightly different:
PHP:
GET /... HTTP/1.1
...
Proxy-Connection: keep-alive
Accept: ...
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 ...
...
The prefix Proxy- is inserted in front of the line "Connection: keep-alive". This prefix is unique and one of the HTTP rules is that the main keywords must always begin at a New Line after Carriage Return ("\r\n"). Knowing that we "fuzzy-program" to intercept the Browser Request and manipulate it so that the Web believes to be connected directly with the browser (not with the Web Proxy):
PHP:
// Conventional Programming
   ByteBuffer wbuf = ByteBuffer.allocateDirect(2048);
   int len = soc.read(wbuf); // read 2KB from Browser SocketChannel
   ((ByteBuffer)wbuf.flip()).get(buf, 0, len);
   SocketChannel webSoc = SocketChannel.open(new InetSocketAddress(webHost, 80)):
   // search for "Proxy-Connection: keep-alive"
   String req = new String(buf, 0, len); // 1. object more with the size of len
   req = req.replace("Proxy-", ""); // 2. and 3. object more (new req, "Proxy-")
   buf = req.getBytes(); // 4. object more (new buf)
   webSoc.write(buf);
   ...
PHP:
// Fuzzy Programming
   ByteBuffer wbuf = ByteBuffer.allocateDirect(2048);
   int len = soc.read(wbuf); // read 2KB from Browser SocketChannel
   ((ByteBuffer)wbuf.flip()).get(buf, 0, len);
   SocketChannel webSoc = SocketChannel.open(new InetSocketAddress(webHost, 80)):
   // search for "Proxy-Connection: keep-alive"
   for (int i 15; i < len; ++i) // start after "XYZ / HTTP/1.1\r >= 15
	 // Checking for "Proxy-" at position
	 // -  0 for byte \n (we know \n is after \r)
	 // -  1 for the letter 'P' we look for P(roxy)
	 // -  5 for the letter 'y' we look for (Prox)y
	 // if all 3 conditions are met we assume that the word "Proxy-" was found.
	 if (buf[i] == '\n' && buf[i+1] == 'P' && buf[i+5] == 'y') { // pos.0,1,5
	   System.arraycopy(buf, i+7, buf, i+1, len - 6); // remove "Proxy-"
	   len -= 6;
	   break;
	 }
   }
   webSoc.write(buf, 0, len);
   ...
The Conventional Programming technique looks quite simple and clear. But it produces 4 following additional objects that must be later "garbage-collected" (bad for Response Time):
  • String req
  • new String Req (due to String immutability)
  • String "Proxy-"
  • old buf (due to array immunity)
The Fuzzy Programming technique does look a bit "confusing", but faster and almost as precise as CP whereby no additional object was needed (less Garbage Collection.)

Fuzzy Programming Rules
FP bases on two assumptions:
  • Assumption that a keyword (e.g. "Proxy-Connection") or a pattern is then found if n letters match the same positions of the keyword/Pattern (0, 1 and 5). Instead of Checking for the whole keyword/pattern (or part of the keyword/pattern) only n unique letters are check at n determinate positions within the keyword. Pattern Recognition bases on such a Fuzzy Programming.
  • Assumption that the Probability (the "rate of Similarity") with n samples (here: the positions) should be so reliable that a deduction could be optimally achieved (up 0.5).


Map Reduce Algorithm (source TheGeekStuff.com)

Google Search Engine (GSE) works with Map Reduce Algorithm or MRA which is no other thing than Fuzzy Programming. MRA calculates the so-called "Term Frequency" or TF which is the frequency (or occurrence) of a term (or an expression, e.g. "Proxy") within a document (or object) and then the Inverse Document Frequency or IDF which is the weight of TF within a heap of documents. With TF-IDF GSE is able to deduct some search results whether they are the same or just similar (e.g. how much similar?) The deduction is based on the calculation "CosineSimilarity".

Some words about CosineSimilarity: We know in the trigonometry that cos(0°) is 1 and cos(90°) is 0. A range between [0, 1]. An angle of 90° indicates a perpendicularity. An angle of 0° signifies a parallelism of objects. Parallelism is in the geometry the similarity between two lines: the CosineSimilarity.
Cosine similarity then gives a useful measure of how similar two documents are likely to be in terms of their subject matter...The technique is also used to measure cohesion within clusters in the field of data mining.
(Wikipedia: CosineSimilarity)

And AI bases on Similarness, Commonness, Approximateness and Relativeness.

Back to our Web Proxy: with FP we can enhance our Web Proxy with more features, for example, a blacklist (of perversive ads), Cookie-Filter, a bridge to other networks (e.g. TOR network), etc. The full codes can be downloaded here (either with ZIP or with JZIP).
 
Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
2,885
1,288
113
(cont.)
Fuzzy Logic

The "pure" lore of Fuzzy Logic is the work of Lotfi Zadeh. FL relies on Mathematical Probability and basses on Set Theory (ST) and for "the ease of vague understanding".

Fuzzy Logic is more real than "computer logic" where the reasoning is always unambiguous. EITHER this (result) OR that (alternative). FL is vague and approximate and there's NO alternative. Example:
Computer Logic (CL)
PHP:
if a == b then c = a // result
else		   c = b // alternative
Fuzzy Logic
PHP:
if a similar b then c is a
  • CL is binary and crisp while FL is ambiguous and vague.
  • CL operators are absolute (==, =, etc) while FL operators are relative (similar, is, etc.)
  • CL object have two distinct states (entities) which are determinable while FL objects possess numerous states which can be vague and overlapping.
  • CL state (or data) is final while FL state (or data) is variable and imprecise.
Because FL is the synonym of Similarity, of Vagueness and of Approximateness FL data are indeterminate, imprecise and so FL variables become ambiguous. Thus Operations with FL data and FL variables are also vague, indeterminate and ambiguous. Such an absoluteness like Equal (==), Greater (>), etc. is too precise in oder to render an "ambiguous state" of an object which is in reality very common (e.g. cold, hot, warm, etc.)

FL Operations must be able to operate on the vague state of FL objects so that any transition between the states could be rendered. Because there's NO distinct transition between different states the ST operations (AND, OR, INVERSE, EXCLUSIVE/INCLUSIVE OR, INVERSE, etc.) can be optimally applied to FL. I don't go into details with Set-Theory operations, but hint you to see the given link above, or to Google for more if you want to do some researches on ST issues. Or THIS.

In order to render numerous states of a FL object FL Data must be a set of various ranges. Example: FL data for temperature:
  • cold is between -10° and +10°,
  • cool is between +8° and +18°,
  • fair is between +15° and +20°,
  • warm is between +18° and +26°,
  • hot is between +24° and +34°,
The data of Temperature can be extended, for example, to: very cold, absolute cold, very hot, boiling hot, etc. The state of each data range is floating between 0.0 to 1.0 (a double or a float[BCOLOR=#ffffff], -1.0 [/BCOLOR]indicates the out of range) and has a form of either a pyramid or a trapezoid as following

The Y-Axis is the "Fuzzy Axis" and the X-Axis is the Sample-Axis. The perpendicular line that cuts the left or right line of the pyramid (or trapezoid) is the sample line. On the Sample-Axis it's the sampled value. The projection of the cutting point on Fuzzy-Axis gives the Fuzzy value. The calculation of the left and right area is called the "defuzzification". The result is then the final outcome of FL operations. Defuzzification can be computed as the Average which can be "fined-grained" or "coarse", or as the Center (or Centroid). An example of a FL operation:

PHP:
if temperature is cool AND AirCon is low then room is cool
As we know, FL does not provide an alternative, hence the outcome is singular: room is cool. Nothing more. Nothing less. And the outcome (room) is also vague and ambiguous: cool between 8° and 18°. AI bases on knowledge and knowledge is big data. Data with complex and intricate ramification. Face or fingerprint or linguistic recognition is for example the most intensive R&D of today. However the results are mediocre. It isn't the inability of researchers and developers, it's the intricate ramification. And ramification can only be rendered by similarness and commonness of some certain traits. With FL we could convey some properties which mirror these features of the observing object. Face recognition appraises and measures, for example, the locations and forms of the eyes, mouth and nose and then deduct the face that has the most similar traits with a reference in an existing repository. Such a work is very time-consuming without FL.

Machine Learning is another field that requires lots of detailed data. Big Data. If Face Recognition is extended to include animal faces more references must be established and validated (supervisor or unsupervisor). The locational details of eyes, mouth and nose must be reworked and categorized (human, animals, birds, insects). FL provides here the base for ML: dog-like (coyote, wolf, chihuahua, etc.), feline (lion, tiger, chetah, etc.) and so on.

Some IT developers think that AI needs AI-oriented language, Front-End requires JS and PHP, etc. They are wrong. All programming languages are suited to code AI applications. The knowhow, not the language itself is the key. JAVA or C/C++ is suited for AI as well. PYTHON is an interpretative language which can directly execute (the source codes) without compiling. And that was the lure to code in PYTHON (by researchers or developers who come from other fields than IT) to see how their AI-Idea works. Hence it's simple to work with "type-less" variables (int, float, double, etc.) because their types will be recognized and adjusted by PYTHON during the interpretative phase. And that can be done with JAVA or C/C++, too, as an AI engine which interprets the AI codes and execute them. Similar to an "AI-Engine" FL does not need a special FL language, but an FL engine that interprets the codes and executes them -exactly as PYTHON does.

Example: the following JAVA codes are the reduced codes of my FL-Engine. The excerpts show you how easily a FL-Engine is implemented in JAVA. The interpretive work may look complicated, but it isn't. It's the basic implementation technology of Interpreter. The FL result is FUZZY. Meaning: vague, imprecise. To attain the "crisp" values FL-Variable must be "defuzzified". The Defuzzification is the weighting of all set samples of FL-Variable. The calculation bases on the left and right area of each sample. If the computing is either
  • fine-grained, or
  • coarse, or
  • centroid
FL depends on FuzzyData and how the states (data) are arranged as pyramid or trapezoid. If the arrangements are contradictory FL could deliver contradictory result, too.

The "ancient" naming for "Machine Learning" is "Automatization". Due to limited memory and mediocre processor "ancient" automatized machine is good, but not as intelligent as the modern "Machine" which can learn. Hence: Machine Learning. As discussed in previous chapter ML is AI that bases on a big repository knowledge. And knowledge is Similarness, Commonness, Approximateness and Relativeness. Such requirements suit the best Fuzzy Logic and FuzzyData is a state which describes an entity and ranges from X to Y -not determinate, but floating.

The given example Room.java is a simplified Machine Learning in both modes: Supervisor and Unsupervisor. With 2 parameters Room can be controlled either by a "Supervisor" or "itself" the Supervisor (Unsupervisor).
PHP:
import fuzzylogic.FuzzyEngine;
// Joe Nartca (C)
// 2 parameters
// - 1st parameter (a double) is the outside temperature
// - 2nd parameter (anything) is for Machine Learning.
// 1 parameter: either a double (outside temperature) or anything (see 2nd parameter)
// NONE: default. -10 Celsius, Supervisor
public class Room {
  public static void main(String... args) throws Exception {
	double outside = -10d;
	boolean auto = false;
	if (args.length > 0) {
	  try {
		outside = Double.parseDouble(args[0]);
		if (args.length == 2) auto = true;
	  } catch (Exception ex) {
		outside = -10d;
		auto = true;
	  }
	}
	// preset Room Temperature between -10 to 35 Celsius
	double temp = outside < -10d? -10d:(outside > 35)? 35:outside;
	...
	FuzzyEngine eng = new FuzzyEngine();
	// Temperature
	eng.defFuzzyData("cold", -100d, 0d, 10d);
	...
	eng.defFuzzyVariable("room");
	eng.addFDtoFV("room", "cold");
	...
	// AirConditioner
	eng.defFuzzyData("min", -10.5d, -8d, -6.5d);
	...
	eng.defFuzzyVariable("AirCon");
	eng.addFDtoFV("AirCon", "min");
	...
	// temp
	eng.defFuzzyVariable("temp");
	eng.addFDtoFV("temp", "cold");
	...
	double rate = 0.0d;
	// the range of values 20d...22d is the "knowledge" (known in repository)
	while (temp < 20d || temp > 22d) {
	  eng.setFVSample("room", temp);
	  if (auto) { // Unsupervisor Machine Learning
		eng.execute("if room is cold then AirCon is max");
		eng.execute("if room is cool then AirCon is high");
		eng.execute("if room is fair then AirCon is OFF");
		eng.execute("if room is warm then AirCon is low");
		eng.execute("if room is hot  then AirCon is min");
		rate = eng.defuzzy("AirCon", 0);
	  }
	  eng.setFVSample("AirCon", rate);
	  eng.execute("if room is cold and AirCon is min then temp is cold");
	  eng.execute("if room is cool and AirCon is min then temp is cool");
	  eng.execute("if room is fair and AirCon is min then temp is fair");
	  eng.execute("if room is warm and AirCon is min then temp is fair");
	  eng.execute("if room is hot  and AirCon is min then temp is warm");
 
	  eng.execute("if room is cold and AirCon is low then temp is cool");
	  eng.execute("if room is cool and AirCon is low then temp is cool");
	  eng.execute("if room is fair and AirCon is low then temp is fair");
	  eng.execute("if room is warm and AirCon is low then temp is fair");
	  eng.execute("if room is hot  and AirCon is low then temp is warm");
 
	  eng.execute("if room is cold and AirCon is OFF then temp is cold");
	  eng.execute("if room is cool and AirCon is OFF then temp is cool");
	  eng.execute("if room is fair and AirCon is OFF then temp is fair");
	  eng.execute("if room is warm and AirCon is OFF then temp is warm");
	  eng.execute("if room is hot  and AirCon is OFF then temp is hot");
 
	  eng.execute("if room is cold and AirCon is high then temp is cool");
	  eng.execute("if room is cool and AirCon is high then temp is fair");
	  eng.execute("if room is fair and AirCon is high then temp is warm");
	  eng.execute("if room is warm and AirCon is high then temp is warm");
	  eng.execute("if room is hot  and AirCon is high then temp is hot");
 
	  eng.execute("if room is cold and AirCon is max then temp is fair");
	  eng.execute("if room is cool and AirCon is max then temp is warm");
	  eng.execute("if room is fair and AirCon is max then temp is warm");
	  eng.execute("if room is warm and AirCon is max then temp is hot");
	  eng.execute("if room is hot  and AirCon is max then temp is hot");
	  temp = eng.defuzzy("temp", 0);
	  if (!auto) { // Supervisor Machine Learning
		rate += temp < 20? 0.05d: -0.05d;
		rate  = rate < -10d? rate + 1d:rate < 10d? rate:rate - 1d;
	  }
	}
	System.out.printf("Outside: %2.2f Celsius, Room: %2.2f Celsius, AirCon: %2.2f\n",
					  outside, temp, rate);
	...
  }
}
The following excerpt shows you how FuzzyLogic Expressions are interpreted and executed.
PHP:
package fuzzylogic;
// Joe Nartca (C)
public class FuzzyEngine {
  public FuzzyEngine( ) {
	...
  }
  public void defFuzzyVariable(String name) {
	...
  }
  // FD pyramid
  public void defFuzzyData(String name, double left, double top, double right) {
	...
  }
  // FD trapezoid
  public void defFuzzyData(String name, double left, double ltop, double rtop, double right) {
	...
  }
  ...
  // Simplify FuzzyEngine with only 2 ST-Operations: AND and OR
  // FL Expresion: if FV1 is FD1 and/or FV2 is FD2 ... then FVr is FDr
  public void execute(String fuzzy) throws Exception {
	String[] terms = fuzzy.split(" ");
	ArrayList<String> fExp = new ArrayList<String>(terms.length);
	for (String s:terms) if (s.length() > 0) fExp.add(s);
	if (fExp.size() < 8) throw new Exception("Incomplete FuzzyExpression:"+fuzzy);
	boolean ok = false; boolean IF = true;// syntax check
	for (int i = 0, mx = fExp.size(); i < mx; ) {
	  if (!fExp.get(i++).equalsIgnoreCase("if") && IF)
		throw new Exception("Expected \"if\" but found:"+fExp.get(i-1));
	  if (!FVs.containsKey(fExp.get(i++)))
		throw new Exception("Expected \"FV\" but found:"+fExp.get(i-1));
	  if (!fExp.get(i++).equalsIgnoreCase("is"))
		throw new Exception("Expected \"is\" but found:"+fExp.get(i-1));
	  if (!FDs.containsKey(fExp.get(i++)))
		throw new Exception("Expected \"FD\" but found:"+fExp.get(i-1));
	  if ((i+1) >= mx) {
		if (!ok) throw new Exception("Invalid Syntax:"+fuzzy);
		break;
	  }
	  String op = fExp.get(i);
	  ok = op.equalsIgnoreCase("then");
	  if (!op.equalsIgnoreCase("and") && !op.equalsIgnoreCase("or") &&
		  !ok) throw new Exception("Expected \"Operator\" but found:"+op);
	  if (i >= mx) throw new Exception("Invalid Syntax:"+fuzzy);
	  IF = false;
	}
	// execute
	IF = true;
	ok = false;
	double result = -1d;
	for (int i = 0, mx = fExp.size(); i < mx; i += 4) {
	  String op = fExp.get(i);
	  String fvName = fExp.get(i+1);
	  String fdName = fExp.get(i+3);
	  FuzzyVariable fv = FVs.get(fvName);
	  if (IF) {
		IF = false;
		result = fv.isFuzzy(fdName);
	  } else if (op.equalsIgnoreCase("then")) {
		if (result != -1d) fv.addPoint(fdName, result);
		return;
	  } else {
		double d = fv.isFuzzy(fdName);
		if (op.equalsIgnoreCase("and")) {
		  if (d < result) result = d;
		} else { // it's an OR
		  if (d > result) result = d;
		}
	  }
	}
  }
  ...
}
Download the room.zip (conventional ZIP file) or room.txt (the JZIP file) and un(j)zip it. The content is:
  • FuzzyData.java
  • FuzzyVariable.java
  • FuzzyEngine.java (reduced from Joe's FuzzyEngine.java)
  • Room.java
and if you've successfully compiled the sources and run the "Room" the Room's output is as following:
PHP:
C:\links\java\AI_FL>javac -g:none -d ./classes Room.java
 
C:\links\java\AI_FL>java Room 100
Outside: 100,00 Celsius, Room: 20,40 Celsius, AirCon: -10,00
 
C:\links\java\AI_FL>java Room 30
Outside: 30,00 Celsius, Room: 21,80 Celsius, AirCon: -9,50
 
C:\links\java\AI_FL>java Room 50
Outside: 50,00 Celsius, Room: 20,40 Celsius, AirCon: -10,00
 
C:\links\java\AI_FL>java Room -50
Outside: -50,00 Celsius, Room: 21,80 Celsius, AirCon: 7,00
 
C:\links\java\AI_FL>java Room -10
Outside: -10,00 Celsius, Room: 21,80 Celsius, AirCon: 7,00
 
C:\links\java\AI_FL>java Room -1
Outside: -1,00 Celsius, Room: 20,40 Celsius, AirCon: 7,00
 
C:\links\java\AI_FL>java Room -1 auto
Outside: -1,00 Celsius, Room: 21,80 Celsius, AirCon: 7,95
 
C:\links\java\AI_FL>
NOTE: next article I'll show you how to implement a full-functional FuzzyEngine (syntax and scripts) which is partly shown here (FuzzyData, FuzzyVariable and simplified FuzzyEngine)
 

Attachments

Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
2,885
1,288
113
3. Repository

"Knowledge is power!" says a proverb. And Francis Bacon said
For knowledge, too, itself is power.
With Knowledge AI knows what AI's doing. And Knowledge must be kept and just-in-time available when AI needs it. Otherwise AI is not lively, but stonedead. I am talking about the AI repository. The brain (of every living species) is an unmatched repository with a lightning fast access. You see a face and know either this face is familar or unknown -less than a blink. That's the power of the brain and that makes every living species superior against any man-made robot -even the skrewdest AI robot.

AI needs also its own Repository -just like every living species. The knowledge can be shared (as information), but not mutually owned. And Repository is a storing vault which is known in IT as the Database (DB). However DB is more or less very simple-structured compared to the structure of the brain. The Relational DB requires keys to access the data which are alas so "normalized" that the objects themselves are irrecognizable. Or Normalization in other words: ACID. And it stands for: Atomicity (in lay language: indivisible), Consistency, Isolation and Durability. For example, the term Isolation alone makes the data unrelated to their surrounding. But Knowledge always correlates with its surrounding. Two eyes, for example, are only recognizable in correlation with a face: human or animal or insect. The consequence is that RDB is very ill-suited as the DB for an AI Repository. Object is object and cannot be "atomized". A car is a complete object and as a car, nota combination of 4 wheels, hood, engine, etc. And the AI world is the same world seen by human (or AI Object-Oriented developers):


(Source: Medium.com)

Knowledge about Vehicle leads to different types of vehicles, the type says about the object and the object indicates its own identity (e.g. sedan, SUV or hatchback). Such an object-reasoning and identifying require a specific object-oriented DB (OODB) or OO-Repository. However commercial OODB is too generic to fit for an AI work. This link gives you an overview about 9 available OODBs. I am myself never involving with any of these OODB. It's in my belief that AI needs OODB which is specialized to each individual AI purposed application. We are not able to cover everything like our brain, hence it's more apt to tailor a specific AI OO-Repository (OOR) for a specific application. It sounds costly and wasteful, but it's in my opinion cheaper and more useful for a long term thinking. Technology evolves and is always a new base for a new technology. AI developers don't need to develop a versatile OOR (like coding with PYTHON -regardless to its interpretative slowness), but see the additional OOR work as a necessary coding work to achieve the most optimal AI application. Mother Nature has showed us that every living species has its own OOR. It isn't a costly-and-wasteful work of Mother Nature.

OOR should be also a "home-made" OODB because a generic AI is impossible and unfeasible. Face or Finger Recognition is, for example, different to DataMining with an AI-search algorithm. Their data are specific and therefore they must be treated differently. Java (or other OOPL) Objects are linguistic object-bound and can only be processed by their own methods written in Java. And that makes sense to build or to develop an OOP-specific OOR. To do that Java Objects need only to be Serialized.

Serialized Java Objects can be stored, fetched and processed by their own "knitted" OODB methods. The schematic of an OOR can be simple (flat file) or complex like a professional OODB (Client-Server). It all depends on the requirements and your IT ability to be capable to develop such an extra work. Coding is already a work. If one learns to code one should learn to design one's own "data container", too. The Object-Oriented Repository.

Implementation

The (OO)DB foundation is simple: IO activities. It's the lowest level. All DB-IO activities rely and to be dependent on the host Operating System and its subsystems (disk, USB, etc.). IO means file IO. File leads to file-system. And you have two choices: use the default file structure or invent your own.

The first option is simple, but it does not give you much room to maneuver. You have to work with OS conventions and OS formats -with all their advantages and disadvantages. The last option requires you, as DB Designer-Implementer, to be creative and inventive. Here you can set your own conventions, your own formats and your own (file) structure. I show you hereunder the last option.

Space saving is disk space saving. When you design an architectural structure of your DB file you need to know:

- How can I save my data optimally? Are there different ways to compress data ? Also, what compress technique should I use ? Does the compression process take more time than a system-conventional WRITE of NON-compressed data?
- How can I read my compressed data as fast a s possible ? Does the decompression process take more time than a system-conventional READ process of NON-compressed data ?
- How can I optimize the compression and decompression process ?
- How can I access the data correctly and efficiently ?

JAVA IO and JAVA NIO both give us in plenty of possibilities to design and to implement our own DB file. First of all, we define our DB file format



An implementation example:
PHP:
  // java.io
  public void open( ) throws Exception {
	if (!(new File(dbName)).exists()) { // create an empty file
	  try (FileOutputStream fo = new FileOutputStream(dbName, false)) {
		fo.close();
	  } catch (Exception ex) {
		throw new Exception("Unable to create OODB:"+dbName);
	  }
	} else try (FileInputStream fi = new FileInputStream(dbName)) {
	  if (fi.available() > 0) { // DB file is NOT empty
		// load and decompress DB data (byte-wise)
		GZIPInputStream gi = new GZIPInputStream(fi);
		ByteArrayOutputStream bo = new ByteArrayOutputStream();
		byte[] rec = new byte[MAX_BUF]; // 64K
		//
		int p;
		for (p = gi.read(rec); p != -1; p = gi.read(rec)) bo.write(rec, 0, p);
		rec = bo.toByteArray();
		gi.close();
		bo.close();
		// cache keys and objects
		p = 0;
		while (p < rec.length) {
		  // compute key length
		  int kL = (int)((rec[p++]&0xFF) << 8)+
				   (int)((rec[p++]&0xFF));
		  // and object length
		  int oL = (int)((rec[p++]&0xFF) << 24)+
				   (int)((rec[p++]&0xFF) << 16)+
				   (int)((rec[p++]&0xFF) << 8)+
				   (int)((rec[p++]&0xFF));
		  // retrieve object
		  byte[] obj = new byte[oL];
		  String k = new String(rec, p, kL);
		  //
		  keys.add(k);
		  System.arraycopy(rec, p+kL, obj, 0, oL);  
		  cache.put(k, obj);
		  p += (kL+oL);
		}
	  }
	} catch (Exception ex) {
	  throw new Exception("Cannot open:"+dbName);
	}
	closed = false;
  }
With Java FileChannel (java.nio)
PHP:
  // java.nio
  public void open( ) throws Exception {
	if (!(new File(dbName)).exists()) {
	  ...
	} else try (FileChannel fc = (new FileInputStream(dbName)).getChannel()) {
	  if (fc.size() > 0) { // DB file is NOT empty
		ByteArrayOutputStream bo = new ByteArrayOutputStream();
		ByteBuffer bb = ByteBuffer.allocateDirect(MAX_BUF);
		// load and decompress DB data (block-wise)
		int p;
		byte[] rec = new byte[MAX_BUF];
		for (p = fc.read(bb); p > 0; p = fc.read(bb)) {
		  ((ByteBuffer)bb.flip()).get(rec, 0, p);
		  bo.write(rec, 0, p);
		  bb.clear();
		}
		fc.close();
		GZIPInputStream gi = new GZIPInputStream(new ByteArrayInputStream(bo.toByteArray()));
		bo.reset(); // clear ByteArrayOutputStream
		for (p = gi.read(rec); p > 0; p = gi.read(rec)) bo.write(rec, 0, p);
		rec = bo.toByteArray();
		gi.close();
		bo.close();
		// cache keys and objects
		p = 0;
		while (p < rec.length) {
		  ....
		}
	  }
	} catch (Exception ex) {
	  throw new Exception("Cannot open:"+dbName);
	}
	closed = false;
  }
The FileChannel implementation applies nio to exploit the bock-wise IO processing. It looks a bit awkward and somehow slow. Yes. It could be slower than the byte-wise implementation. But as long as the DB file is small (less than 100KB) FileChannel is at disadvantage. The situation reverses when the DB file size increases (up more than 100KB).

Using GZIPInputStream and GZIPOutputStream we could reduce the used disk space up to 67 % (see Screenshot.)


With this OOR structure we are able to develop lots of derivatives what I've done with the JZIP application (see ZIP or GNU-ZIP)
 
Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
2,885
1,288
113
Before I shows some examples let's complete the self-built OO Repository and talk about Design & Implementation of Data Integrity and Data Consistency. The odb-package can be downloaded hereunder.

In Part I we design and implement our own file system with the following entry-format (or record format):

PHP:
0			2			  6				k		   n
+-----------+--------------+----------------+-------------+
! KeyLength ! ObjectLength ! Key/ObjectName ! Object	  !
+-----------+--------------+----------------+-------------+
2 bytes KeyLength: max. 64K
4 bytes ObjectLength: max 2GB
k bytes Key/ObjectName: max. 64 KB
n bytes Object: max. 2GB
  • Key or ObjectName-Length (2 bytes) indicates the size of object name. Max. 64KB or a string of 64KB
  • ObjectLength (4 bytes) is the size of the object itself in bytes. Each object can have max. 2 GB (BigData)
  • Key/ObjectName is the readable name of the following object. It must be unique.
  • Object with the size of max. 2G Bytes. An object could be a serialized JAVA object, or a byte-array which could be object of other OOPL, a content of an image/sound file, or an URL link to another object, etc.
Note: if the byte array of an object is the content of a file (e.g. image) it's wise to use its name as the key/objectName (e.g. JoePicture.jpg). However, image/sound files are usually big it's better to keep them where they are and use their name as reference (absolute path or URL if they are on the WEB).

Then we apply the GZIP compress technology to our dbFile in order to reduce the physical size of our dbFiles (up to 60% -see screenshot of Part I).

Depending on the "UserRight" open() automatically create an empty DB file if there isn't a dbFile under the given dbName. The UserRight could be: ReadOnly, ReadWrite and ReadWriteDelete. ReadOnly users can only open existed dbFile. If the dbFile under the given dbName exists its content will be loaded into an ODMS cache (HashMap). Nevertheless, the caching mechanism (or algorithm) should be modified or devised if your dbFile grows too big (e.g. hundreds of objects with the size of some GB) or your DB server's scarce of RAM capacity.

Data Integrity and Data Consistency depend on the way how we access the data, how we make sure that the data are consistent, persistent (i.e. integrity) within a multiuser environment. Before we go into details let's complete the accessing mechanism of our DB file. We set the next convention for the naming storage: for the ease of use we decide that our DB files should be grouped in a common directory -says: [current_Directory]/obb/dbName] or [current_Directory]\obb\[dbName] on WINDOWS, where dbName should be without suffix. Example: dbFile is "employee".

PHP:
   current/working directory is: C:\links\odb
   the common directory will be: C:\links\odb\odb
   our dbFile for "employee" is: C:\links\odb\odb\employee
The complete ODMS.java. This implemenation is an example showing you how the basics of an Object Data Management System is implemented. You can enhance or modified it to suit your requirement.

Note: However, you have to mention the copyright (C) of the author Joe Nartca and Congdongjava.com as the source. The author and Congdongjava.com are NOT responsible for any loss or damage of your data. You're on your own.

Multiuser Environment - Networking ODB-Server


Fig.4

In a multiuser environment users, or to be more precise applications, compete for resources. Either all resources are held exclusively by ONE user or they are shared by all users. The first option is simple to implement. User who needs a resource gets it or has to wait for it till it's release. No locking or unlocking mechanism is required. But the processing time becomes unpredictable. Due to waiting for resources some applications appear to be hung up. Such a design & implementation is obsolete.

Sharing resources? Yes. The most optimal option. Modern design bases on the Client-Server principle: a joint server serves all clients and distributes the competing resources equally among the competitors. However, the design and implementation vary from application to application. But the principle stays unchanged, Design & Implementation could be vary from application to application. A flexible design is to build two common frameworks on each side: client and server. The frameworks are in our design & implementation the APIs (Application Programming Interfaces).

Fig.4 depicts the scenario of communication between Clients and Server. The arrows give you a raw idea how the joint APIs work: which API works with what API. SocketChannel is the communication base between Clients and Server (block-wise). Let's follow the communicating path of our Design & Implementation. The imaginary sequence:
  • UserX sends a connection request to ODBServer (host/port).
  • ODBServer accepts UserX's request and spawns an ODBWorker which is run by JAVA ExecutorService in its StealingPool (parallel & concurrent. More: Concurrent & Parallel Programming With CPU-Affinity or read THIS)
  • ODBWorker represents its "partner UserX" on the Server side and follows its directions to deal with the specified database (or dbFiles) by commanding ODManager to handle the requests
  • ODManager invokes in its turn the in-question ODMS (one of its controlled ODMSes) to process the IO requests
  • ODMS does the IOs and hands out the result to ODManager
  • ODWorker finally picks the result (see ODBResult) and sends it back to its partner UserX.

Fig.5 shows you how Data are cached and how they are managed. On open all data are uploaded into ODMS Cache (JAVA HashMap). All subsequent IOs happen within this Cache and in a random manner of Key-Data: also with key as the accessing index. This way (HashMap) provides the fastest IO-access time.

Our Client-Server design bases on the following APIs:
  • ODBService. This API provides its users the possibility to design their own GUI-driven ODB Server. The invocation is simple: 3 parameters for the constructor (String config, port number and an OutputStream for the logger). If you have WEB users as clients you have to Port-Forwarding the given port. More about Port-Forwarding you'd consult your MODEM's manuals (or click HERE or Google with the keyword "Port Forwarding").
  • ODBWorker. This API is started internally by ODBService. If an user connects to ODBService it spawns an ODBWorker which is executed in StealingPool and works exclusively for this user.
  • ODBResult. The interchange of outcomes between Server and Client is wrapped in ODBResult. Again, this API is a template. You should enhance or modify this POJO to cope with your personal requirements.
An example of ODBResult.java
PHP:
package odb;
/**
 POJO Template for OODB-result after a transaction
 <br>byte array obj: reserved
 <br>boolean done, updated, existed, locked reflect the DBConnect Methods
 <br>String array Keys
 <br>String data, err, dbName
 @author Joe Nartca (C)
*/
public class ODBResult implements java.io.Serializable {
  private static final long serialVersionUID = 1L;
  public java.util.ArrayList<String> list;
  public String keys[], err = "";
  public boolean bool = false;
  public byte[] obj;
}
ODBService employs ODBManager API (which manages the IOs with ODMS API) and passes it as a mutual parameter to its children (ODBWorkers). To ensure Data Integrity & Consistency, and to shield OODB from unwanted abuses Server and Client should be usually protected by Password and (unique) UserID. I include an API UserList into this odb package. However, UserList requires some protective encrypting-decrypting algorithm. The best protection is an algorithm which is known only by you yourself -nobody else! Hence you have to devise some encrypting-decrypting algorithm for yourself.

The EnDecrypt template is as following:
PHP:
package odb;
/**
An Encrypt-Decrypt template -to be developed.
@author Joe Nartca (C)
*/
public class EnDecrypt {
  /**
	decrypt an encrypted String
	@param inp encrypted String
	@return decrypted String (null if invalid inp)
  */
 public static String decrypt(String inp) {
   // your decrypting codes
   return inp;
 }
  /**
	encrypt() a String. A dynamic algorithm using a random number as starting key
	<br>and 10-logarithm as the base to compute and to encrypt the given string
	@param inp to be encrypted String
	@return encrypted String (null if invalid inp)
  */
  public static String encrypt(String inp) {
	// your encrypting codes
	return inp;
  }
}
To ease your (temporary) work I include in the package my own EnDecrypt API (144 bits algorithm without source) so that you can immediately work with the package. Of course, you can use this API as long as you want. It's free of charge. (The link to download odb.jar will be available at the end of this series). The include userlist contains 1 user. Password: joey, userID: joe.

The implementation of UserList API bases on a GZIP file (that is in the common odb directory). Each user entry has the following format:
PHP:
+------------------------------+---+-----------+
!  encrypted Password:UserID   ! @ ! Privilege !
+------------------------------+---+-----------+
  • The encrypted string (flexible length) is the concatenation of a Password string of any length and an UserID of any length, separated by a colon :))
  • @, 1 byte, is the separator between encrypted PW:uID and user's privilege.
  • Privilege, 1 byte, is one integer digit: 0 for ReadOnly, 1 for ReadWrite and 2 for ReadWriteDelete. Higher number should be reserved for further expansion (e.g. for remote User privilege)
An example of API UserList
PHP:
package odb;
 
import java.io.*;
import java.util.ArrayList;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
/**
UserList offers 3 following functions:
<br>1. AutoCreate a predefined gzip-file "userlist" if this file doesn't exist
<br>2. Login Validation when login is true
<br>3. UserList maintenance with add/delete
<br>Privileg: Read only (0), ReadWrite (1), ReadWriteDelete (2)
<br>Entry Format: Password:UserID@Privileg
@author Joe Nartca (C)
*/
public class UserList {
  /**
  Contructor.
  @param uFile  String, userlist file (abs.Path).
  @exception Exception thrown by JAVA (IOs, etc.)
  */
  public UserList(String uFile) throws Exception {
	this.uFile = uFile;
	uList = new ArrayList<String>();
	if ((new File(uFile)).exists()) {
	  GZIPInputStream gin = new GZIPInputStream(new FileInputStream(uFile));
	  ByteArrayOutputStream bao = new ByteArrayOutputStream();
	  byte[] bb = new byte[65536];
	  //
	  int n;
	  while ((n = gin.read(bb)) > 0) bao.write(bb, 0, n);
	  StringBuilder buf = new StringBuilder(new String(bao.toByteArray()));
	  gin.close();
	  bao.close();
	  //
	  String[] lst = buf.toString().split("\r\n");
	  for (String s : lst) {
		n = s.indexOf("@");
		uList.add(EnDecrypt.decrypt(s.substring(0, n))+s.substring(n));
	  }
	}
	mod = false;
  }
  /**
  isUser
  @param pw  String, User Password
  @param uid String, User Login ID
  @return boolean false if user is unknown
  */
  public boolean isUser(String pw, String uid) {
	for (String u : uList) if (u.indexOf(pw+":"+uid+"@") == 0) return true;
	return false;
  }
  /**
  getUserPrivilege()
  @param pw  String, User Password
  @param uid String, User Login ID
  @return int 0: readOnly, 1: ReadWrite, -1 if user is unknown
  */
  public int getUserPrivilege(String pw, String uid) {
	if (isValid(pw, uid) && isUser(pw, uid)) {
	  for (String u : uList) if (u.indexOf(pw+":"+uid+"@") == 0) {
		return u.charAt(u.length()-1) & 0x0F;
	  }
	}
	return -1;
  }
  /**
  add new User into userlist
  @param pw  String, User Password
  @param uid String, User Login ID
  @param right int, 0: only Read, 1: ReadWrite, 2: ReadWriteDelete
  @return boolean true: user was added
  */
  public boolean addUser(String pw, String uid, int right) {
	if (!isValid(pw, uid) || isUser(pw, uid)) return false;
	if (right > 1) right = 2;
	else if (right < 0)  right = 0;
	uList.add(pw+":"+uid+"@"+right);
	mod = true;
	return true;
  }
  /**
  update UserID or Password in userlist
  @param pwOld  String, User Password
  @param uidOld String, User Login ID
  @param pwNew  String, User Password
  @param uidNew String, User Login ID
  @return boolean false if invalid old pw and/or uid
  */
  public boolean updateUser(String pwOld, String uidOld,
							String pwNew, String uidNew) {
	if (!isValid(pwOld, uidOld) || !isValid(pwNew, uidNew) ||
		!isUser(pwOld, uidOld) || isUser(pwNew,uidNew)) return false;
 
	for (String u : uList) if (u.indexOf(pwOld+":"+uidOld+"@") == 0) {
	  uList.set(uList.indexOf(u), pwNew+":"+uidNew+u.substring(u.indexOf("@")));
	  mod = true;
	  return true;
	}
	return false;
  }
  /**
  update UserPrivilege in userlist
  @param pw  String, User Password
  @param uid String, User Login ID
  @param newPriv  int, privilege. 0: Read, 1: ReadWrite, 2: ReadWriteDelete
  @return boolean false if invalid pw and/or uid
  */
  public boolean updateUser(String pw, String uid, int newPriv) {
	if (isValid(pw, uid)  && isUser(pw, uid)) {
	  if (newPriv > 1) newPriv = 2;
	  else if (newPriv < 0) newPriv = 0;
	  for (String u : uList) if (u.indexOf(pw+":"+uid+"@") == 0) {
		uList.set(uList.indexOf(u), pw+":"+uid+"@"+newPriv);
		mod = true;
		return true;
	  }
	}
	return false;
  }
  /**
  update UserID or Password in userlist
  @param pwOld  String, User Password
  @param uidOld String, User Login ID
  @param pwNew  String, User Password
  @param uidNew String, User Login ID
  @param newPriv  int, privilege. 0: Read, 1: ReadWrite, 2: ReadWriteDelete
  @return boolean false if invalid old pw and/or uid
  */
  public boolean updateUser(String pwOld, String uidOld,
							String pwNew, String uidNew, int newPriv) {
	if (!isValid(pwOld, uidOld) || !isValid(pwNew, uidNew) ||
		!isUser(pwOld, uidOld) || isUser(pwNew,uidNew)) return false;
 
	if (newPriv > 1) newPriv = 2;
	else if (newPriv < 0) newPriv = 0;
	for (String u : uList) if (u.indexOf(pwOld+":"+uidOld+"@") == 0) {
	  uList.set(uList.indexOf(u), pwNew+":"+uidNew+"@"+newPriv);
	  mod = true;
	  return true;
	}
	return false;
  }
  /**
  delete User
  @param pw  String, User Password
  @param uid String, User Login ID
  @return boolean false: unknown uid/pw
  */
  public boolean deleteUser(String pw, String uid) {
	if (isValid(pw, uid) && isUser(pw, uid)) {
	  for (String u : uList) if (u.indexOf(pw+":"+uid+"@") == 0) {
		uList.remove(u);
		mod = true;
		return true;
	  }
	}
	return false;
  }
  /**
  getUserList a list of all Users
  @return String array contains all users as "pw:uid@privilege"
  <br>where [privilege] = 0 Read only, 1 ReadWrite, 2: ReadWriteDelete
  */
  public ArrayList<String> getUserList( ) {
	return uList;
  }
  /**
  Save userlist
  @exception Exception thrown by JAVA (IO, etc.)
  */
  public void save( ) throws Exception {
	if (mod) {
	  GZIPOutputStream gzip = new GZIPOutputStream(new FileOutputStream(uFile));
	  StringBuilder sb = new StringBuilder(uList.size());
	  for (String s : uList) {
		int n = s.indexOf("@"); // EnDecrypt.encrypt(pw:uid) only
		sb.append(EnDecrypt.encrypt(s.substring(0, n))+s.substring(n)+"\r\n");
	  }
	  gzip.write(sb.toString().getBytes());
	  gzip.flush( );
	  gzip.close( );
	}
  }
  //
  private boolean isValid(String pw, String uid) {
	if (pw == null || uid == null || pw.length() == 0 || uid.length() == 0) return false;
	return true;
  }
  //--------private area
  private boolean mod;
  private String uFile;
  private ArrayList<String> uList;
}
Hereunder is a simple ODBServer using the API ODBService. Firstly we have to create an ODB configuration: the config.txt
PHP:
#
# this is an example
# ODB_PATH is the reserved Keyword
#
ODB_PATH=C:/links/odb/odb
#
PHP:
import odb.*;
// Joe Nartca
public class ODBServer  {
  /**
  @param port int, Server's port
  @exception Exception thrown by JAVA
  */
  public ODBServer(String config, int port) throws Exception {
	System.out.println("CRTL-C to quit");
	// instantiate ODBService using System.out as logger
	ODBService bs = new ODBService(config, port, System.out);
	System.out.println(".....ODB Server is running...");
  }
  // Localhost, port 9999 (default) or your own port number
  public static void main(String... a) throws Exception {
	new ODBServer(a.length > 0? a[0]:"config.txt", 9999);
  }
}
The infinite loop
PHP:
while(running) pool.execute(new ODBWorker(dbSvr.accept(), odMgr));
indicates that a PoolThread ODBWorker will be spawned directly after the waiting server (dbSvr) received an incoming Client request. The spawned thread represents the connected client to work with ODBManager. A Client can only access ODBServer (or ODBService) using ODBConnect (Client API -see Part IV). To do that it has to send a connection request with
  • the dbName it wants to connect to
  • password
  • userID
The password and userID will be encrypted before they are sent to ODB Server. If the authentication failed ODBWorker sends an error message to its "client partner" and waits for the next "retry".

Note: As usual, if you modify or enhance the sources you have to mention the copyright (C) of the author Joe Nartca and Congdongjava.com as the source. The author and Congdongjava.com are NOT responsible for any loss or damage of your data. You're on your own.
 

Attachments

Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
2,885
1,288
113
Some Example

For FuzzyLogic I'll show you more on the next posting "How to implement a full-functional FuzzyEngine (syntax and scripts) and How to integrate FL into and with JAVA".

FuzzyLogic Language (FFL) Overview
PHP:
		  FuzzyLogic Keywords, Operatots, Syntax and Operations
 
FuzzyVariables and FuzzyData must be registered before FuzzyExpressions or
FuzzyScript can be executed.
 
The keyword 'this' refers to JavaObject which is the Container of FuzzyEngine
(i.e. where FE is instantiated). Registered and Unregistered can be referred by
FuzzyEngine. if Unregistered JavaObject is referred, but all the Object's
contents (i.e. objects, primitives) are in their initial states (or values) and
they could be null or 0.
 
Registered JavaObjects must be instantiated, before they are registered.
Otherwise Exception will arise when they are referred.
 
Reference to JavaVariables (or fields): JavaObjectName:JavaVariableName, or
										this:JavaVariableName
Reference to JavaMethods: JavaObjectName.JavaMethodName(Parm1, ...) or
						  this.JavaMethodName(Parm1, ...)
 
------------------Register FuzzyData and FuzzyVariable--------------------------
[FDname:left, top, right]
[FDname:left, ltop, rtop, right]
{FVname:FDname, FDname, ..., FDname}
--------------------------Buil-in Functions-------------------------------------
clear FV		  // clear all FV-FuzzyData of FV
reset FV		  // FV and all its FDs
--------------------------Operations--------------------------------------------
FV = val		  // FV sample = val
FV + val		  // + FV sample with val
FV - val		  // - FV sample from val
FV * val		  // * FV sample by val
FV / val		  // / FV sample by val
---------------Operation with JavaVariable--------------------------------------
FV = MyApp:temp   // set FV sample with JavaVariable temp of JavaClass MyApp
MyApp:temp = val  // set JavaVariable to val
MyApp:temp + val  // add JavaVariable with val
MyApp:temp - val  // sub JavaVariable from val
MyApp:temp * val  // mul JavaVariable by val
MyApp:temp / val  // div JavaVariable by val
--------------------------Statements--------------------------------------------
if FV is FD then FVr is FDr	 // assign valuated result to FDr of FVr
if (FV is FD) then FVr is FDr
if FV1 is FD1 or FV2 is FD2 then FV3 is FD3
if (FV1 is FD1) or (FV2 is FD2) then FV3 is FD3
if (FV1 is FD1 or FV2 is FD_2) and FV3 is FD3 then FV4 is FD4
if ((FV1 is FD1 or FV2 is FD_2) and FV3 is FD3) then FV4 is FD4
--------------------------Statements--------------------------------------------
if FV some FD ...			 // fuzzySample**2 of FD
if FV very FD ...			 // SQRT(fuzzySample) of FD
if FV not  FD ...			 // (1-fuzzySample) of FD
if ... then FVr some FDr	  // assign result**2 to FDr of FVr
if ... then FVr very FDr	  // assign SQRT(result) to FDr of FVr
if ... then FVr not  FDr	  // assign 1-result to FDr of FVr
-----------------------Operators and Syntax-------------------------------------
// starts a comment and ends at end of the Line
+ - * / = : .
 
---------------------------Keywords---------------------------------------------
this
reset
clear
some
very
not
and
or
if
then
is
--------------------------------------------------------------------------------
and an example written in FFL script
PHP:
// Temperature states
[cold: -20d, -5d, 5d]
[cool: 2d, 10d, 20d]
[fair: 19d, 20d, 22d]
[warm: 21d, 23d, 25d]
[hot:  24d, 30d, 50d]
// AirCon regulator states
[min: -10d, -6d, -3d]
[low: -4d, -2d, 0]
[OFF: -0.5d, 0, 0.5d]
[high: 0d, 2d, 4d]
[max:  3d, 6d, 10d]
// FuzzyVariables
{AirCon: min, low, OFF, high, max }
{room: cold, cool, fair, warm, hot }
{temp: cold, cool, fair, warm, hot }
// set Samples to Java Global Variables
clear AirCon
room = AHouse:temp
if room is cold then AirCon is max
if room is cool then AirCon is high
if room is fair then AirCon is OFF
if room is warm then AirCon is low
if room is hot  then AirCon is min
// set AirCon with defuzzied FV
AHouse.print("Time to run")
AHouse.print("Room:", room)
AirCon = AHouse.adjust(AirCon)
//AirCon = AirCon
// FuzzyExpressions
if room is cold and AirCon is min then temp is cold
if room is cool and AirCon is min then temp is cold
if room is fair and AirCon is min then temp is cold
if room is warm and AirCon is min then temp is cool
if room is hot  and AirCon is min then temp is fair
  
if room is cold and AirCon is low then temp is cold
if room is cool and AirCon is low then temp is cold
if room is fair and AirCon is low then temp is cool
if room is warm and AirCon is low then temp is fair
if room is hot  and AirCon is low then temp is warm
  
if room is cold and AirCon is OFF then temp is cold
if room is cool and AirCon is OFF then temp is cool
if room is fair and AirCon is OFF then temp is fair
if room is warm and AirCon is OFF then temp is warm
if room is hot  and AirCon is OFF then temp is hot
  
if room is cold and AirCon is high then temp is cool
if room is cool and AirCon is high then temp is fair
if room is fair and AirCon is high then temp is warm
if room is warm and AirCon is high then temp is hot
if room is hot  and AirCon is high then temp is hot
  
if room is cold and AirCon is max then temp is fair
if room is cool and AirCon is max then temp is warm
if room is fair and AirCon is max then temp is hot
if room is warm and AirCon is max then temp is hot
if room is hot  and AirCon is max then temp is hot
// end of FuzzyExpression
AHouse.print("End of Fuzzy. RoomTemp:", temp)
AHouse.print()
AHouse:temp = temp  // set Java temp with FV temp
and the Java
PHP:
import fuzzylogic.FuzzyEngine;
public class AHouse {
  private static double temp, step, outside;
  //
  private void print( ) {
	System.out.printf("Invoke by FuzzyScript: Outside %2.2f Celsius, Room %2.2f Celsius, AirCon %2.2f\n",
					  outside, temp, step);
  }
  private void print(String msg) {
	System.out.println("Invoke by FuzzyScript: "+msg);
  }
  private void print(String msg, double d) {
	temp = d; // set the new temperature
	System.out.println("Invoke by FuzzyScript: "+msg+d);
  }
  private static double adjust(double d) {
	double x = temp < 5d? 1d:temp < 20d?0.5d:temp < 25d? -0.5:-1d;
	step = d + x;
	System.out.printf("Invoke by FuzzyScript: Defuzzy step:%2.2f -> new step:%2.2f by temp:%2.2f\n", d, step, temp);
	return step;
  }
  public static void main(String... args) throws Exception {
	String script = "Ahouse.txt";
	outside = -10d;
	if (args.length > 0) {
	  try {
		outside = Double.parseDouble(args[0]);
		if (args.length > 1) script = args[1];
	  } catch (Exception ex) {
		script = args[0];
		outside = -10d;
	  }
	} else {
	  java.util.Random ran = new java.util.Random();
	  step = (double)ran.nextInt(10);
	  outside = (double)ran.nextInt(40);
	}
	step = 0d;
	// preset Room Temperature between -10 to 35 Celsius
	temp = outside < -10d? -10d:(outside > 32)? 32:outside;
	// FuzzyLogic..............................
	FuzzyEngine eng = new FuzzyEngine(this,"AHouse");
	// registered java Variable step of this class
	eng.regJavaVariable("AHouse:temp", temp);
	eng.regJavaVariable("AHouse:step", step);
	// the values between 20d and 22d are the so-called LEARNED values
	while (temp < 20d || temp > 21d) {
	  eng.execute(script);
	}
  }
}
and the result is as following:
PHP:
C:\JFX\Fuzzy\Test>java AHouse 1 ahouse.txt
Invoke by FuzzyScript: Time to run
Invoke by FuzzyScript: Room:1.0
Invoke by FuzzyScript: Defuzzy step:6,20 -> new step:7,20 by temp:1,00
Invoke by FuzzyScript: End of Fuzzy. RoomTemp:20.6
Invoke by FuzzyScript: Outside 1,00 Celsius, Room 20,60 Celsius, AirCon 7,20
 
C:\JFX\Fuzzy\Test>
The given ZIP file contains 5 files showing you how [BCOLOR=#ffffff]Map Reduced Algorithm[/BCOLOR] works in "Data Mining" in a "big" directory. It's written in SWING and self-explained. The "MRA Terms" are the the searching patterns. For example: to search in a directory files which contain the string "FuzzyLogic" the MRA Term is the word "FuzzyLogic".

If anyone of you, mon ami Nancru and all CDJ members, can post your question(s) here and I'll try my best to give you a satisfactory answer. Up 12. Mai 2019 - end Juni 2019 I'll be absent here (on vacation in north Vietnam and in south China).
 

Attachments

Sửa lần cuối: