The Rise of AI Through Machine Learning
Artificial Intelligence

The Rise of AI Through Machine Learning

James Gray 18 September 2023

Introduction

The history of artificial intelligence has seen many peaks and troughs. After an initial burst of optimism in the 1950s, progress slowed. Rules-based expert systems were unable to mimic the flexibility of human cognition. But in recent decades, a technique called machine learning has enabled a dramatic revival.

Powerful statistical learning algorithms allowed computers to find patterns in huge datasets. The results often surpassed human-crafted logic and rules. Machine learning delivered major advances across natural language processing, computer vision, speech recognition and more. AI began achieving state-of-the-art results in an astonishing range of tasks. This is the story of how machine learning led to the 21st century boom in artificial intelligence.

The Limitations of Symbolic AI

In the early decades of artificial intelligence research, scientists focused on symbolic AI, which involved encoding facts, rules, and logic to mimic human reasoning. This approach relied on predefined representations and expert systems that contained heuristics from specialists in narrow domains like medical diagnosis.

While symbolic AI showed promise in restricted environments, it ultimately proved inadequate for general intelligence. There were two primary reasons for its downfall. First, human cognition relies on vast, interconnected webs of concepts, background knowledge, and commonsense reasoning. It was nearly impossible to capture the full complexity of human thought using formal logical systems and symbols. The breadth and nuance of human reasoning could not be reduced to neat, predefined rules.

Second, symbolic AI systems were extremely brittle. They lacked the flexibility and fault tolerance that characterizes human thinking. Even minor deviations from expected inputs or scenarios would cause hard failures. Without the ability to gracefully handle uncertainty, exceptions, and noise, these systems could not cope with the messiness of the real world.

The rigidity of symbolic AI stems from its reliance on fixed representations and logic. It has no mechanism for handling novelty or unpredictability. Even limited domains like medical diagnosis have too much variability for a symbolic system to capture all possible symptoms and disease combinations using hand-coded knowledge.

To achieve more human-like intelligence, AI researchers realized they needed to move beyond symbolic reasoning. New approaches would embrace probability, uncertainty, and learning from statistical relationships in data. Rather than trying to encode all knowledge explicitly, systems should be able to extract patterns from experience in a more flexible way.

This paradigm shift laid the foundations for modern techniques like machine learning and neural networks. By leveraging the strengths of both symbolic and sub-symbolic AI, today’s systems can better withstand noise while explaining their reasoning and behavior. The limitations of purely symbolic methods paved the way for more powerful hybrid approaches to artificial intelligence.

The Rise of Machine Learning

Traditional symbolic AI relied on human experts to manually code rules and logic. But this approach struggled to scale as the complexity of real-world tasks exceeded what programmers could reasonably encode by hand. An alternative paradigm known as machine learning emerged as a solution.

Rather than teaching computers explicitly through rigid programming, machine learning algorithms are trained using data. By exposing an algorithm to many labeled examples, it can automatically learn the trends and patterns in the data without any explicit rules provided by human programmers. This enables the system to extract insights that would be impossible to code manually.

Machine learning has its origins in pattern recognition research from the 1950s. Early work focused on the perceptron – the simplest type of artificial neural network. While limited to linear models, perceptrons introduced the power of learning from data. In the 1960s and 70s, simple statistical machine translation systems were also developed.

But machine learning did not take off in earnest until the 1980s, when backpropagation was discovered. This algorithm enabled multi-layer neural networks to be trained efficiently. Neural networks could now model complex nonlinear relationships for problems like image recognition and natural language processing.

Other important developments include support vector machines in the 1990s for classification and regression, tree search algorithms for game-playing, and convolutional neural networks which drove breakthroughs in computer vision starting in 2012.

As datasets grew exponentially in size and compute power massively increased, machine learning began to dominate AI. It achieved superhuman performance at tasks ranging from playing games like chess and Go to language translation and automated driving. Unlike brittle symbolic AI, machine learning systems can contend with the noise and complexity of the real world by learning from experience.

Today, machine learning powers many of our everyday technologies and tools. It has enabled key innovations in fields such as computer vision, speech recognition, robotics, genomics, healthcare, finance, and more. The automatic pattern recognition capabilities of machine learning will continue to transform how software systems are constructed in the 21st century.

Core Machine Learning Algorithms

Many different machine learning algorithms contributed to the AI revolution. Here are some of the most important categories:

Neural Networks

Neural networks are brain-inspired algorithms structured as interconnected nodes called artificial neurons. Data flows through the network, with each layer detecting different features. Weights between nodes are adjusted during training to improve the results. Neural networks can model extremely complex non-linear relationships. Architectures like deep neural networks and convolutional neural networks led to huge advances in computer vision and speech recognition.

Support Vector Machines

Support vector machines are supervised learning models used for classification and regression. They separate data points into different classes using a hyperplane. The algorithm maximizes the margin between the hyperplane and the nearest data points on each side. Kernel functions can be used to project the data into higher dimensions to find optimal separating hyperplanes. SVMs are effective on high-dimensional sparse datasets.

Ensemble Methods

Ensemble methods combine multiple learning algorithms to improve overall predictive performance. Techniques like bagging, boosting and stacking train a collection of base models, then have them vote on the output. By accounting for biases and variance across different algorithms, ensembles can produce more accurate solutions. Random forests of decision trees are a popular ensemble method.

Clustering

Clustering algorithms group unlabelled data based on similarity. K-means clustering partitions data points into k clusters by minimizing intra-cluster variance. Hierarchical clustering builds a hierarchy of clusters using linkage criteria. Density-based clustering like DBSCAN separates high-density and low-density regions. Clustering helps find inherent structures in unstructured datasets.

Dimensionality Reduction

Real-world data often contains redundant features. Dimensionality reduction simplifies data by projecting it onto a lower-dimensional space. Principle component analysis uses orthogonal transformations to convert data into uncorrelated principal components. Other techniques like linear discriminant analysis, t-SNE and autoencoders are also used. This removes noise and improves computational performance.

Probabilistic Graphical Models

Graphical models like Bayesian networks and Markov random fields compactly represent multivariate probability distributions over variables. They encode dependencies between variables using graphs. Nodes represent random variables while edges depict conditional dependencies. Querying these probabilistic models can infer likelihoods even with uncertain, incomplete data. They are applicable to a wide range of prediction and pattern recognition problems.

The Impact of Machine Learning

Applying these machine learning algorithms led to remarkable progress across many AI applications:

  • Computer vision – Convolutional neural networks analyzed images and video at human levels and beyond. Self-driving cars used computer vision for navigation and obstacle avoidance.
  • Natural language processing – Vast amounts of text data was used to train algorithms on machine translation, sentiment analysis and language generation. Chatbots leveraged NLP to converse naturally with humans.
  • Robotics – Reinforcement learning enabled robots to learn control policies by trial-and-error. Robots mastered skills like grasping objects, walking and opening doors without explicit programming.
  • Gaming – AI agents defeated professional players at complex games like chess, Go, poker and Dota 2. Tree search, neural networks and reinforcement learning drove the development of superhuman game-playing algorithms.
  • Recommendation systems – Collaborative filtering and matrix factorization algorithms automatically suggested relevant products, media and connections in e-commerce, social networks and entertainment services.
  • Business analytics – Predictive analytics, clustering and anomaly detection extracted actionable insights from industry data to drive decision making. Fraud detection systems analyzed transactions for illegal activity.

The list goes on. From drug discovery to predictive maintenance, machine learning delivered breakthrough capabilities across every industry. Its flexible, data-driven approach proved immensely powerful and adaptable. This statistical revival propelled artificial intelligence out of stagnation and into an exciting new era.

Conclusion

Machine learning has been the engine driving the recent resurgence of AI. Its statistical, pattern-recognition approach overcame key limitations holding back symbolic AI. Given enough data, machine learning algorithms can model extremely complex functions without human intervention. Neural networks in particular have delivered incredible advances in perception and decision making.

The 21st century will be defined by humans and AIs working together to augment our collective intelligence. Machine learning finally provided artificial intelligence with a methodology equal to the complexity of the real world.