A Brief History Of Machine Learning _ Dr. jaideep ganguly 33a6m
This document was ed by and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this report form. Report 3i3n4
Overview 26281t
& View A Brief History Of Machine Learning _ Dr. jaideep ganguly as PDF for free.
More details 6y5l6z
- Words: 3,055
- Pages: 10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
A Brief History of Machine Learning Published on November 2, 2016
Dr. Jaideep GangulyFollow
Director of Software Development, Amazon
305
16
63
The buzz around Machine Learning and Deep Learning prompted me to trace the history of Artificial Intelligence at MIT and elsewhere and take stock of the current state of Machine Learning. Before we get started, a quick overview of some ; Learning is acquisition of knowledge, discovery is the observation of a new phenomena and invention is the process of making something new. Learning is necessary for invention but is not a sufficient condition for innovation. Machine Learning as it stands today, does not invent but it does discover patterns in large quantities of data. In particular, Deep Neural Networks, have attracted the imagination of many because of some interesting solutions it offers in all three channels text, speech and images. Incidentally, most deep neural nets are rather wide and are usually not more than ten layers deep. So the name should have really been "wide neural nets" but the word "deep" has stuck.
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim", said Dijkstra. It is more interesting to understand the evolution of Machine Learning
how did it start, where are we today and where do we go from here. The human brain is a remarkable thing; it has enabled us understand science and advance mankind. The idea of mimicking the human brain or even improving the human cognitive functions is an alluring one and is an objective of Artificial Intelligence research. But we are not even close inspite of a century of research. However, it continues to have a major hold on our imagination given the potential of the rewards.
It seems that about 50,000 years ago, after we have been around for about a hundred thousand years or so, palaeontologists believe that some of us, possibly just a few thousand, were able to deal with symbols. This is a major step in evolution. Noam Chomsky thinks we were then able to create a new concept from 2 existing ideas or concepts without damaging or limiting the existing concepts. Around 350 BC, Aristotledevised syllogistic logic, the first formal deductive reasoning system to model the way humans think about their world and reason with it. 2,000 years later, Bertrand Russeland Alfred Whitehead published Principia Mathematica that laid down the foundations for a formal representation of Mathematics. John McCarthy, who championed the cause of mathematical logic in AI, was Aristotle of his day. In 1942, Alan Turing showed that any form of mathematical reasoning could be processed by a machine. https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
1/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
By 1967, Marvin Minsky declared that "within a generation, the problem of creating Artificial Intelligence would substantially be solved". Clearly we are not there yet, attempts to build systems with
first order logic as described by early philosophers failed because of lack of computing power, inability to deal with uncertainty and lack of large amounts of data.
In 1961, Minsky published "Steps towards Artificial Intelligence" where he talked about search, matching, probability, learning and it was quite visionary. Turing told us that it was possible to make a machine intelligent and Minsky told us how. In 1986, Minsky wrote the highly influential book "The Society of Mind", 24 centuries after Plato wrote "Politeia"; Minsky was Plato of his days. Minsky taught us to think about heuristic programming, McCarthy wanted us to use logic to the extreme, Newel wanted to build cognitive models of problem solving and Simon believed that when we see something that is complicated in behavior, it is more of a consequence of a complex environment rather than because of a complex thinker. Thereafter, a number of model backed systems were built. Terry Winograd built a model backed system for dialog understanding, Patrick Winston built another model backed system for learning and Gerald Sussmanbuilt a model backed system for understanding blocks. During the same era Roger Schank believed that understanding stories is the key to modeling human intelligence. David Marr, who is best known for his work on vision, treated vision as an information processing system. Marr's trilevel hypothesis in cognitive science comprised of a computational level what does the system do, an algorithmic level how does the system do and a physical level how is the system physically realized, e.g., in the case of biological vision, what neural structures and neuronal activities implement the visual system.
In the 1980s, the expert systems were of great interest and focused on knowledge and inference mechanisms. While these systems did a pretty good job in their domains, they were narrow in specialization and were difficult to scale. The field of AI was defined as computers performing tasks that were specifically thought of as something only humans can do. However, once these systems worked, they were no longer considered to be AI! For example, today the best chess players are routinely defeated by computers but chess playing is no longer really considered as AI! McCarthy referred to as the "AI effect". IBM's Watson is a program at a level such as that of a human expert but it is not certainly not the first one. Fifty years ago Jim Slagle's symbolic integration program at MIT was a tremendous achievement. Nevertheless, it is very hard to build a program that has "common sense" and not just narrow domains of knowledge.
Today, at the core is the debate between logic inspired and neural network inspired paradigms for cognition. LeCun, Bengio and Hinton state that succinctly in a review paper in Nature, dated 28th May 2015, as "The issue of representation lies at the heart of the debate between the logicinspired and the neuralnetworkinspired paradigms for cognition. In the logicinspired paradigm, an instance of a symbol is something for which the only property is that it is either identical or nonidentical to other symbol instances. It has no internal structure that is relevant to its use; and to reason with symbols, https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
2/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
they must be bound to the variables in judiciously chosen rules of inference. By contrast, neural networks just use big activity vectors, big weight matrices and scalar nonlinearities to perform the type of fast ‘intuitive’ inference that underpins effortless commonsense reasoning".
Rosenblatt is credited with the concept of Perceptrons, “a machine which senses, recognizes, re, and responds like the human mind” as early as in 1957 but in a critical book written in 1969 by Marvin
Minsky and Seymour Papert showed that Rosenblatt’s original system was painfully limited, literally blind to some simple logical functions like XOR. In the book they said: "... our intuitive judgment that the extension (to multilayer systems) is sterile". This intuition was incorrect and the field of “Neural
Networks” pretty much disappeared! Geoff Hinton built more complex networks of virtual neurons that allowed a new generation of networks to learn more complicated functions (like the exclusiveor that had bedeviled the original Perceptron). Even the new models had serious problems though. They learned slowly and inefficiently and couldn’t master even some of the basic things that children do. By the late 1990s, neural networks had again begun to fall out of favor. In 2006, Hinton developed a new technique that he dubbed deep learning, which extends earlier important work by Yann LeCun. Deep learning’s important innovation is to have models learn categories incrementally, attempting to nail down lowerlevel categories (like letters) before attempting to acquire higherlevel categories (like words).
In April 2000, in a seminal work published in Nature by Mriganka Sur, et. al, at MIT's laboratory for brain and cognitive sciences, the authors were able to successfully “rewire" brains in very young mammals, inputs from the eye were directed to brain structures that normally process hearing. The animal's auditory cortex successfully interpreted input from its eyes. But it didn't do the job as well as the primary visual cortex would have, suggesting that while the brain's plasticity, or ability to adapt, is enormous, it is limited by genetic preprogramming. Environmental input, while key to the development of brain function, does not "write on a blank slate". This addresses an ageold question – is the brain is genetically programmed or shaped by
environment? It is a dramatic evidence of the ability of the developing brain to adapt to changes in the external environment, and speaks to the enormous potential and plasticity of the cerebral cortex – the seat of our highest abilities. This provided some theoretical underpinning to the neural net computation theory.
Deep neural nets spawned a subset known as Recurrent Neural Nets which were an attempt to model sequential events. Vector Machines, logistic regression, feedforward networks have proved very useful without explicitly modeling time. But the assumption of independence precludes modeling long range dependencies. DNNs were also helped by the emergence of GPUs which enabled parallelism as much of computation in DNN is intrinsically parallel in nature. RNNs are connectionist models with the ability to selectively information across sequence steps while processing sequential data one element at a time. They can model input and/or output consisting of sequences of elements that are not independent. However, learning with recurrent networks is difficult. For standard feedforward networks, the optimization task is NP https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
3/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
complete. Learning with recurrent networks is challenging due to the difficulty of learning longrange dependencies. Problems of vanishing and exploding gradients occur when back propagating errors across many time steps. In 1997, Hochreiter and Schmidhuber introduced the Long Short Term Memory (LSTM) model to overcome vanishing gradients. LSTMs have been proven to be remarkable in speech and handwriting recognition. Similarly, another variation of the Deep Net Model is the Convolution Neural Network (CNN) that has been very successful in classifying images.
In conclusion, we have come a long way. Deep Nets appear to be very promising in some areas although they are computationally very expensive. However, deep learning is only part of the larger challenge of building intelligent machines. It lacks ways of representing causal relationships, have no obvious ways of performing logical inferences, and is still a long way from integrating abstract knowledge, such as information about what objects are, what they are for, and how they are typically used. The most powerful A.I. systems, like Watson use techniques like deep learning as just one element in a very complicated ensemble of techniques, ranging from the statistical technique of Bayesian inference to deductive reasoning.
The name "Machine Learning" is indicative of the potentials that it can possibly achieve in the future. In the next article, I will talk about what problems in the industry that can be solved with the current state of technology and its evolution in the next couple of years.
Report this
Dr. Jaideep Ganguly
Director of Software Development, Amazon
Follow
1 post
16 comments Recommended Leave your thoughts here…
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
4/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
3d
Sabyasachi "Sky" Basu Thought Leader of Digital Innovation Jaideep, a great introduction to Machine Learning.
"However, once these systems worked, they were no longer considered to be AI!"
In mid80s I met the grand dame of Cognitive Science and AI Margaret Boden (https://en.wikipedia.org/wiki/Margaret_Boden) in a conference. I asked her a very basic question "What is AI?". And her answer was (I am paraphrasing)
"In natural science, when we do not know how a natural phenomenon works we call it 'metaphysics', when we know we call it 'physics', 'chemistry', 'biology'. Similarly in Computer Science when we don't know how to solve a problem we call it 'AI', and when we can solve it fairly well we then call it 'database', 'computer graphics', 'networking'."
I thought then and still think there is a wonderful wisdom in that explanation. Like
Reply
12
1
2d
Dr. Jaideep Ganguly Director of Software Development, Amazon Well said. And that is why AI gets thrown under the bus from time to time, many of the things that we take for granted today are because of the work done in this field. Like
Reply
1
4d
KPM Das Director, Cybersecurity and Trust, India at Cisco Systems Brilliant primer for a status take part one..... recall having made little progress with "rules" based reasoning in the late eighties on PROLOG and LISP implementations, this wave of ML seems prodigious ...............look forward to part two
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
5/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
Like
Reply
4
1
4d
Dr. Jaideep Ganguly Director of Software Development, Amazon Thanks KPM! Like
Reply
2
2d
Ron Kaplan Vice President and Chief Scientist at A9.com (Amazon) Hi Jaideep, Very nice summary. I would point out that the definition of "learning" as acquiring "knowledge" begs a deeper question: what is knowledge?
On one view, knowledge is information that can be demonstrated or put to multiple uses, inspected in the course of making inferences, and transmitted to others so that they can also make use of it. On that view, learning in the sense of acquiring knowledge is different from learning in the sense of acquiring a skill. If I have learned the location of a particular business, I can demonstrate that I have that knowledge by going there, but I can also tell someone else how to go there, estimate the distance from other locations, etc. But when I have learned the skill, say, of riding a bike. I can't communicate what I have learned to someone else in a way that will enable them to also ride a bike. They have to learn it by themselves, presumably by lots of their own practice.
At least in their current state of development, I think what deep learning systems acquire and represent in their network models is more like a skill than transmissible and inspectable knowledge. I can deliver a model to somebody else so that they can execute it (which is a good thing), but there isn't much else that I or they can do beyond that. If I have a good speech recognition model, I can recognize speechbut I can't say much if anything about how that is accomplished (which might be useful for other applications).
The knowledge/skill (or habit) distinction harkens back to the cognitive reaction to the behaviorist tradition in psychology, the argument (by Chomsky and others) that it is important to recognize https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
6/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
the difference between acquiring knowledge in the form of internal, manipulable mental representations and acquiring the ability to map from inputs to correlated outputs (stimuli to responses, acoustic waves to wordsequences, images to classes, etc.) And since we both have now mentioned Chomsky, of relevance to this kind of discussion might be Chomsky's review of Skinner's Verbal Behavior, written in the 50's. The technology is new, but the issues have been around for awhile.
Best, Ron Like
Reply
6
1d
Stéphane Pisani Étudiantchercheur | En rédaction de mémoire I like this 'genealogic way' of describing machine learning. I have now a better understanding of all concepts related to machine learning. The tracing of their lineages and history could be easily captured in a mind map format for future reference. Thanks a lot Jaideep! Like
Reply
3
2d
Raja Boddu. Ph.D Principal & Head, R&D at Lenora College of Engineering Good one Like
Reply
3
2d
Subbu Mandiga Manager III, Software Development at Amazon Awesome read ...looking forward for the followup article Like
Reply
2
3d
Anand Iyer Senior Test Architect https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
7/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
When you talked about the socalled 'debate between logic inspired and neural network inspired paradigms for cognition', it cleared up a longstanding question on my mind. and, that was this how do I distinguish between AI and a software program, or is there a difference at all? especially, given that the former also requires software programming at some level.
At least, now I know even the experts are split between the two!
Thanks for the enlightenment, Dr. Jaideep Ganguly. Looking forward to the next part! Like
Reply
2
3d
Vaibhav Mittal Compensation and Benefits Specialist Global Competency Center at Xerox Priyal Mittal Like
Reply
1
2d
Ravindra Prasad Head of Engineering and Technology at Tesco Great insights into Machine learning. Best article I read so far on Machine learning. Thank you Jaideep. Like
Reply
1
1d
Dr. Ravi Vadlamani Professor & Head, Center of Excellence in Analytics, Institute for Development and Research in… Very well captured. However, one must not forget Ivakhnenko's work on Group Method of Data Handling (GMDH) network of 1965, which is the first 'deep learning' neural network, even though he did not coin that word. Like
Reply
1
4d https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
8/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
Sharmila Ganguly Design Consultant Excellent research Like
Reply
1
3d
Souvik Kar Software Development Engineer II at Amazon Awesome read Like
Reply 3d
Sanjay Hora Founder, TechArda.com nice background information on ML. cant wait to see the next post :) Like
Reply 3d
Methil Sreekumar consultant ,strategic telecom management, general istration A very interesting article ,chronologically laid out to pick up lost threads . At this time , I have only one to comment, is evolution of AI prompting or driving human intelligence to pursue unimaginable feats /heights and maintains the proverbial gap with the former lagging behind. Looking forward for the next sequel to flow. Methil Sreekumar Like
Reply 2d
Shreedhar Torgal Technologist | Enterprise Architect | IT Strategy | cognitive Computing | Program management | CxO… Good one.. Like
Reply
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
9/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
2h
Ravi Venkatesan Research Director at Systems Research Corporation Thanks for sharing. Like
Reply
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
10/10
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
A Brief History of Machine Learning Published on November 2, 2016
Dr. Jaideep GangulyFollow
Director of Software Development, Amazon
305
16
63
The buzz around Machine Learning and Deep Learning prompted me to trace the history of Artificial Intelligence at MIT and elsewhere and take stock of the current state of Machine Learning. Before we get started, a quick overview of some ; Learning is acquisition of knowledge, discovery is the observation of a new phenomena and invention is the process of making something new. Learning is necessary for invention but is not a sufficient condition for innovation. Machine Learning as it stands today, does not invent but it does discover patterns in large quantities of data. In particular, Deep Neural Networks, have attracted the imagination of many because of some interesting solutions it offers in all three channels text, speech and images. Incidentally, most deep neural nets are rather wide and are usually not more than ten layers deep. So the name should have really been "wide neural nets" but the word "deep" has stuck.
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim", said Dijkstra. It is more interesting to understand the evolution of Machine Learning
how did it start, where are we today and where do we go from here. The human brain is a remarkable thing; it has enabled us understand science and advance mankind. The idea of mimicking the human brain or even improving the human cognitive functions is an alluring one and is an objective of Artificial Intelligence research. But we are not even close inspite of a century of research. However, it continues to have a major hold on our imagination given the potential of the rewards.
It seems that about 50,000 years ago, after we have been around for about a hundred thousand years or so, palaeontologists believe that some of us, possibly just a few thousand, were able to deal with symbols. This is a major step in evolution. Noam Chomsky thinks we were then able to create a new concept from 2 existing ideas or concepts without damaging or limiting the existing concepts. Around 350 BC, Aristotledevised syllogistic logic, the first formal deductive reasoning system to model the way humans think about their world and reason with it. 2,000 years later, Bertrand Russeland Alfred Whitehead published Principia Mathematica that laid down the foundations for a formal representation of Mathematics. John McCarthy, who championed the cause of mathematical logic in AI, was Aristotle of his day. In 1942, Alan Turing showed that any form of mathematical reasoning could be processed by a machine. https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
1/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
By 1967, Marvin Minsky declared that "within a generation, the problem of creating Artificial Intelligence would substantially be solved". Clearly we are not there yet, attempts to build systems with
first order logic as described by early philosophers failed because of lack of computing power, inability to deal with uncertainty and lack of large amounts of data.
In 1961, Minsky published "Steps towards Artificial Intelligence" where he talked about search, matching, probability, learning and it was quite visionary. Turing told us that it was possible to make a machine intelligent and Minsky told us how. In 1986, Minsky wrote the highly influential book "The Society of Mind", 24 centuries after Plato wrote "Politeia"; Minsky was Plato of his days. Minsky taught us to think about heuristic programming, McCarthy wanted us to use logic to the extreme, Newel wanted to build cognitive models of problem solving and Simon believed that when we see something that is complicated in behavior, it is more of a consequence of a complex environment rather than because of a complex thinker. Thereafter, a number of model backed systems were built. Terry Winograd built a model backed system for dialog understanding, Patrick Winston built another model backed system for learning and Gerald Sussmanbuilt a model backed system for understanding blocks. During the same era Roger Schank believed that understanding stories is the key to modeling human intelligence. David Marr, who is best known for his work on vision, treated vision as an information processing system. Marr's trilevel hypothesis in cognitive science comprised of a computational level what does the system do, an algorithmic level how does the system do and a physical level how is the system physically realized, e.g., in the case of biological vision, what neural structures and neuronal activities implement the visual system.
In the 1980s, the expert systems were of great interest and focused on knowledge and inference mechanisms. While these systems did a pretty good job in their domains, they were narrow in specialization and were difficult to scale. The field of AI was defined as computers performing tasks that were specifically thought of as something only humans can do. However, once these systems worked, they were no longer considered to be AI! For example, today the best chess players are routinely defeated by computers but chess playing is no longer really considered as AI! McCarthy referred to as the "AI effect". IBM's Watson is a program at a level such as that of a human expert but it is not certainly not the first one. Fifty years ago Jim Slagle's symbolic integration program at MIT was a tremendous achievement. Nevertheless, it is very hard to build a program that has "common sense" and not just narrow domains of knowledge.
Today, at the core is the debate between logic inspired and neural network inspired paradigms for cognition. LeCun, Bengio and Hinton state that succinctly in a review paper in Nature, dated 28th May 2015, as "The issue of representation lies at the heart of the debate between the logicinspired and the neuralnetworkinspired paradigms for cognition. In the logicinspired paradigm, an instance of a symbol is something for which the only property is that it is either identical or nonidentical to other symbol instances. It has no internal structure that is relevant to its use; and to reason with symbols, https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
2/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
they must be bound to the variables in judiciously chosen rules of inference. By contrast, neural networks just use big activity vectors, big weight matrices and scalar nonlinearities to perform the type of fast ‘intuitive’ inference that underpins effortless commonsense reasoning".
Rosenblatt is credited with the concept of Perceptrons, “a machine which senses, recognizes, re, and responds like the human mind” as early as in 1957 but in a critical book written in 1969 by Marvin
Minsky and Seymour Papert showed that Rosenblatt’s original system was painfully limited, literally blind to some simple logical functions like XOR. In the book they said: "... our intuitive judgment that the extension (to multilayer systems) is sterile". This intuition was incorrect and the field of “Neural
Networks” pretty much disappeared! Geoff Hinton built more complex networks of virtual neurons that allowed a new generation of networks to learn more complicated functions (like the exclusiveor that had bedeviled the original Perceptron). Even the new models had serious problems though. They learned slowly and inefficiently and couldn’t master even some of the basic things that children do. By the late 1990s, neural networks had again begun to fall out of favor. In 2006, Hinton developed a new technique that he dubbed deep learning, which extends earlier important work by Yann LeCun. Deep learning’s important innovation is to have models learn categories incrementally, attempting to nail down lowerlevel categories (like letters) before attempting to acquire higherlevel categories (like words).
In April 2000, in a seminal work published in Nature by Mriganka Sur, et. al, at MIT's laboratory for brain and cognitive sciences, the authors were able to successfully “rewire" brains in very young mammals, inputs from the eye were directed to brain structures that normally process hearing. The animal's auditory cortex successfully interpreted input from its eyes. But it didn't do the job as well as the primary visual cortex would have, suggesting that while the brain's plasticity, or ability to adapt, is enormous, it is limited by genetic preprogramming. Environmental input, while key to the development of brain function, does not "write on a blank slate". This addresses an ageold question – is the brain is genetically programmed or shaped by
environment? It is a dramatic evidence of the ability of the developing brain to adapt to changes in the external environment, and speaks to the enormous potential and plasticity of the cerebral cortex – the seat of our highest abilities. This provided some theoretical underpinning to the neural net computation theory.
Deep neural nets spawned a subset known as Recurrent Neural Nets which were an attempt to model sequential events. Vector Machines, logistic regression, feedforward networks have proved very useful without explicitly modeling time. But the assumption of independence precludes modeling long range dependencies. DNNs were also helped by the emergence of GPUs which enabled parallelism as much of computation in DNN is intrinsically parallel in nature. RNNs are connectionist models with the ability to selectively information across sequence steps while processing sequential data one element at a time. They can model input and/or output consisting of sequences of elements that are not independent. However, learning with recurrent networks is difficult. For standard feedforward networks, the optimization task is NP https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
3/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
complete. Learning with recurrent networks is challenging due to the difficulty of learning longrange dependencies. Problems of vanishing and exploding gradients occur when back propagating errors across many time steps. In 1997, Hochreiter and Schmidhuber introduced the Long Short Term Memory (LSTM) model to overcome vanishing gradients. LSTMs have been proven to be remarkable in speech and handwriting recognition. Similarly, another variation of the Deep Net Model is the Convolution Neural Network (CNN) that has been very successful in classifying images.
In conclusion, we have come a long way. Deep Nets appear to be very promising in some areas although they are computationally very expensive. However, deep learning is only part of the larger challenge of building intelligent machines. It lacks ways of representing causal relationships, have no obvious ways of performing logical inferences, and is still a long way from integrating abstract knowledge, such as information about what objects are, what they are for, and how they are typically used. The most powerful A.I. systems, like Watson use techniques like deep learning as just one element in a very complicated ensemble of techniques, ranging from the statistical technique of Bayesian inference to deductive reasoning.
The name "Machine Learning" is indicative of the potentials that it can possibly achieve in the future. In the next article, I will talk about what problems in the industry that can be solved with the current state of technology and its evolution in the next couple of years.
Report this
Dr. Jaideep Ganguly
Director of Software Development, Amazon
Follow
1 post
16 comments Recommended Leave your thoughts here…
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
4/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
3d
Sabyasachi "Sky" Basu Thought Leader of Digital Innovation Jaideep, a great introduction to Machine Learning.
"However, once these systems worked, they were no longer considered to be AI!"
In mid80s I met the grand dame of Cognitive Science and AI Margaret Boden (https://en.wikipedia.org/wiki/Margaret_Boden) in a conference. I asked her a very basic question "What is AI?". And her answer was (I am paraphrasing)
"In natural science, when we do not know how a natural phenomenon works we call it 'metaphysics', when we know we call it 'physics', 'chemistry', 'biology'. Similarly in Computer Science when we don't know how to solve a problem we call it 'AI', and when we can solve it fairly well we then call it 'database', 'computer graphics', 'networking'."
I thought then and still think there is a wonderful wisdom in that explanation. Like
Reply
12
1
2d
Dr. Jaideep Ganguly Director of Software Development, Amazon Well said. And that is why AI gets thrown under the bus from time to time, many of the things that we take for granted today are because of the work done in this field. Like
Reply
1
4d
KPM Das Director, Cybersecurity and Trust, India at Cisco Systems Brilliant primer for a status take part one..... recall having made little progress with "rules" based reasoning in the late eighties on PROLOG and LISP implementations, this wave of ML seems prodigious ...............look forward to part two
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
5/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
Like
Reply
4
1
4d
Dr. Jaideep Ganguly Director of Software Development, Amazon Thanks KPM! Like
Reply
2
2d
Ron Kaplan Vice President and Chief Scientist at A9.com (Amazon) Hi Jaideep, Very nice summary. I would point out that the definition of "learning" as acquiring "knowledge" begs a deeper question: what is knowledge?
On one view, knowledge is information that can be demonstrated or put to multiple uses, inspected in the course of making inferences, and transmitted to others so that they can also make use of it. On that view, learning in the sense of acquiring knowledge is different from learning in the sense of acquiring a skill. If I have learned the location of a particular business, I can demonstrate that I have that knowledge by going there, but I can also tell someone else how to go there, estimate the distance from other locations, etc. But when I have learned the skill, say, of riding a bike. I can't communicate what I have learned to someone else in a way that will enable them to also ride a bike. They have to learn it by themselves, presumably by lots of their own practice.
At least in their current state of development, I think what deep learning systems acquire and represent in their network models is more like a skill than transmissible and inspectable knowledge. I can deliver a model to somebody else so that they can execute it (which is a good thing), but there isn't much else that I or they can do beyond that. If I have a good speech recognition model, I can recognize speechbut I can't say much if anything about how that is accomplished (which might be useful for other applications).
The knowledge/skill (or habit) distinction harkens back to the cognitive reaction to the behaviorist tradition in psychology, the argument (by Chomsky and others) that it is important to recognize https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
6/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
the difference between acquiring knowledge in the form of internal, manipulable mental representations and acquiring the ability to map from inputs to correlated outputs (stimuli to responses, acoustic waves to wordsequences, images to classes, etc.) And since we both have now mentioned Chomsky, of relevance to this kind of discussion might be Chomsky's review of Skinner's Verbal Behavior, written in the 50's. The technology is new, but the issues have been around for awhile.
Best, Ron Like
Reply
6
1d
Stéphane Pisani Étudiantchercheur | En rédaction de mémoire I like this 'genealogic way' of describing machine learning. I have now a better understanding of all concepts related to machine learning. The tracing of their lineages and history could be easily captured in a mind map format for future reference. Thanks a lot Jaideep! Like
Reply
3
2d
Raja Boddu. Ph.D Principal & Head, R&D at Lenora College of Engineering Good one Like
Reply
3
2d
Subbu Mandiga Manager III, Software Development at Amazon Awesome read ...looking forward for the followup article Like
Reply
2
3d
Anand Iyer Senior Test Architect https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
7/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
When you talked about the socalled 'debate between logic inspired and neural network inspired paradigms for cognition', it cleared up a longstanding question on my mind. and, that was this how do I distinguish between AI and a software program, or is there a difference at all? especially, given that the former also requires software programming at some level.
At least, now I know even the experts are split between the two!
Thanks for the enlightenment, Dr. Jaideep Ganguly. Looking forward to the next part! Like
Reply
2
3d
Vaibhav Mittal Compensation and Benefits Specialist Global Competency Center at Xerox Priyal Mittal Like
Reply
1
2d
Ravindra Prasad Head of Engineering and Technology at Tesco Great insights into Machine learning. Best article I read so far on Machine learning. Thank you Jaideep. Like
Reply
1
1d
Dr. Ravi Vadlamani Professor & Head, Center of Excellence in Analytics, Institute for Development and Research in… Very well captured. However, one must not forget Ivakhnenko's work on Group Method of Data Handling (GMDH) network of 1965, which is the first 'deep learning' neural network, even though he did not coin that word. Like
Reply
1
4d https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
8/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
Sharmila Ganguly Design Consultant Excellent research Like
Reply
1
3d
Souvik Kar Software Development Engineer II at Amazon Awesome read Like
Reply 3d
Sanjay Hora Founder, TechArda.com nice background information on ML. cant wait to see the next post :) Like
Reply 3d
Methil Sreekumar consultant ,strategic telecom management, general istration A very interesting article ,chronologically laid out to pick up lost threads . At this time , I have only one to comment, is evolution of AI prompting or driving human intelligence to pursue unimaginable feats /heights and maintains the proverbial gap with the former lagging behind. Looking forward for the next sequel to flow. Methil Sreekumar Like
Reply 2d
Shreedhar Torgal Technologist | Enterprise Architect | IT Strategy | cognitive Computing | Program management | CxO… Good one.. Like
Reply
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
9/10
11/7/2016
A Brief History of Machine Learning | Dr. Jaideep Ganguly | Pulse | LinkedIn
2h
Ravi Venkatesan Research Director at Systems Research Corporation Thanks for sharing. Like
Reply
https://www.linkedin.com/pulse/briefhistorymachinelearningdrjaideepganguly?trk=hpfeedarticletitlecomment
10/10
Related Documents 3h463d
A Brief History Of Ibuprofen 1r5u3p
November 2019 53
A Brief History Of Journalism 24295q
December 2019 60
A Brief History Of Tourism 2h2q14
October 2021 0
A Brief History Of Tourism 2h2q14
November 2019 216
A Brief History Of Measurement 5q546y
October 2022 0
A Brief History Of English j5r5z
November 2019 75More Documents from "Ripan Sarkar" 6u3940
4p345f
December 2019 18
Skripsi Modul Rancang Bangun Mesin Pengampelas Kayu 6l2y2v
June 2020 8
Pedoman-dapodik-paud-2018.pdf 2zrn
December 2019 28
Qanun Gampong 2016 a2p44
April 2020 23
Dtsi 22j32
November 2019 39