DaDaDa 2016 - Deep Convolutional Neural Networks in industrial applications

Abstract

This presentation introduces deep convolutional neural networks (ConvNets) for industrial inspection applications, particularly surface defect detection in manufacturing. Beginning with traditional machine learning's limitations—expert-driven handcrafted features using HOG, SIFT, and wavelet methods—the talk establishes why deep learning represents a paradigm shift in artificial intelligence. The fundamental insight, borrowed from Andrew Ng, reveals that while traditional ML algorithms saturate with increasing data volumes, deep learning scales continuously with data—making it essential for the big data era. The presentation explains neural network fundamentals from 1950s perceptrons through the critical 2006 breakthrough when Geoffrey Hinton's stacked autoencoder pre-training solved the vanishing gradient problem that had prevented deep network training. Modern techniques including dropout regularization and ReLU activation functions now enable reliable training of networks with millions of parameters. The architecture shifts from fully-connected networks (which create parameter explosion—200 million parameters for 1000x1000 images) to convolutional networks using local receptive fields and weight sharing (reducing to hundreds of parameters). Through systematic experiments on publicly available defect databases, the speaker demonstrates depth versus width trade-offs: adding layers (depth investigation from 3 to 9 convolutional layers) yielded only 1.4% performance improvement, while adding neurons per layer (width investigation) gained 5% improvement. This reveals that simple geometric defects like scratches don't require extensive hierarchical abstraction—problem complexity should match architecture complexity rather than blindly maximizing depth. The benchmark results show their general-purpose network, trained only on raw pixel data with zero defect-specific knowledge, achieves state-of-the-art performance beating specialized handcrafted computer vision algorithms designed explicitly for defect detection. The presentation emphasizes three stages of ConvNets (convolution, pooling, nonlinear activation) and critical data preparation methodology including patch extraction, augmentation through rotation/mirroring/shearing, and essential data shuffling for multiclass problems. The speaker concludes that deep learning elevates machine learning from low-level pixel manipulation to high-level architecture design, requiring sufficient training data and massively parallel GPU hardware, with particular effectiveness for problems exhibiting hierarchical feature structure.

About the speaker

Daniel Weimer is a researcher at Volkswagen Data Labs specializing in deep learning applications for industrial manufacturing, with particular focus on computer vision and automated quality inspection systems. His work applies state-of-the-art deep convolutional neural networks to practical industrial problems including surface defect detection across various materials and manufacturing contexts. Rhymer's research investigates fundamental questions of neural network architecture optimization—systematically exploring depth versus width trade-offs and developing methodologies for matching network complexity to problem characteristics. At Volkswagen Data Labs, he leads experimental initiatives benchmarking deep learning approaches against traditional computer vision methods, demonstrating how end-to-end learning from raw data can surpass handcrafted feature engineering even without domain-specific knowledge.

Transcript summary

Daniel Weimer from Volkswagen Data Labs presents deep convolutional neural networks with focus on industrial inspection applications, particularly surface defect detection. Through experiments on publicly available defect databases, he demonstrates how deep learning achieves state-of-the-art results without handcrafted features—learning directly from raw pixel data. The key insight: while traditional machine learning saturates with more data, deep learning scales continuously, making it essential for the big data era.

Opening and Audience Check

"Everybody, my name is Daniel Rhymer. I'm from the Volkswagen Data Labs and today I have the pleasure to present some of the work in the field of deep convolutional neural networks with a focus on industrial applications." Right at the beginning, a question: "Who's actively working in their daily work with deep learning?" The speaker indicates he'll start from the beginning, explaining the basic pipeline of machine learning to motivate deep learning, then set focus on deep convnets in industrial inspection—"one of the most important things in manufacturing." He'll present experimental results and, very importantly, limits of deep learning, then conclude.

Traditional Machine Learning Pipeline

The traditional machine learning pipeline: if you want to detect a car, for example, what are you doing? First of all, you have to find an appropriate set of features that describe the car—let's say the shape, the color, or whatever. Then you take these features and feed them into a classifier. This classifier is trained with historical data—standard approach. What might be the next step? Let's see the features. First, you can use a histogram of oriented gradients (HOG), which is a gradient-based method. You can use MSER or SIFT algorithms. You can use wavelet-based features—that's also a good tool for face detection. The same goes for 3D. These are all low-level features—why low-level? "Because we use basically the gradient information, which is a very basic information."

Mid-Level Features: State-of-the-Art Until 2012

What happened until 2012: based on this low-level feature extractor, people put a mid-level feature extractor on it. This means they took the low-level features, and based on these, they had an additional layer with some kind of k-nearest neighbor search, for example, or an autoencoder or things like that to transform low-level features into a mid-feature-level dimension. Then they took these mid-level features to feed the classifier in the same fashion. This was state-of-the-art until 2012.

The Problem: Expert-Driven Feature Engineering

But there's one problem with it. First of all, it still depends on the handcrafted feature representation. "So this means an expert has to define the features." This means it's a very expert-driven kind of machine learning approach. This took a lot of time—it takes a lot of experts and a lot of time to find the best set of features for a given problem.

The Deep Learning Revolution

But now we're in the era of deep learning. "Maybe anywhere in the universe there might be a better set of features compared to the features we already have. We now have the techniques of deep learning to find the features. So let's do it." The speaker wants to describe why it makes absolutely sense: "Take the traditional approach and let me say—I call it the better approach—this huge technique of machine learning, this very powerful technique is only applied in the last step when it comes to model generation, when it comes to building the classification model. All steps before are completely expert-driven. The generation of the features, for example, is expert-driven. At least approximately no machine learning is included here. Machine learning is only included in the very, very last step, even in the better approach with the mid-level features."

Deep Learning: End-to-End Learning

In contrast, deep learning also includes the generation of features into the learning approach. This means the basis is raw data, and we learn the features and the model in one closed framework. "This is the basic idea of deep learning."

The Ultimate Result Example

Another example—what might be the ultimate result of a machine learning algorithm applied to this image? You see some persons, you see a ball, you see it in—the ultimate result might be "the 8th of July in a stadium when Mario Götze, 88th minute, made Germany the champion of the world in football." But only based on this image, the ultimate result might be "four persons playing football in a stadium." This is basically the information which is in this image.

Why Deep Learning Works Better

Imagine standard algorithms—standard computer vision algorithms. You need very different kinds and a lot of algorithms to find all these different things in an image. You need, for example, HOG descriptor to template the persons. You need, for example, a color histogram to see that it's grass. You need, for example, a Hough transformation to find the ball, which is a round piece of object. Deep learning is able to calculate all these different steps in one closed framework, starting from raw data, raw pixel data, and transform it in a nonlinear fashion (you'll see this in a minute) to more complex features until the highest kind of information. This is basically the content of the image: four persons playing football.

The Basic Question of Deep Learning

The basic question of deep learning is: can we learn powerful features directly from data without the need of human interaction, without the need of a process expert who describes features manually? And the answer is yes we can, and we use deep learning.

Second Motivation: Why a New Paradigm?

Before going more into details, the speaker wants a second motivation for deep learning. "So why do we need actually a new paradigm in artificial intelligence or machine learning? We have powerful techniques. We have great techniques—statistical learning theory, support vector machines, and things like that. Why do we need deep learning? Why do we need a new paradigm?"

The Andrew Ng Insight

The answer is this slide: "I borrowed it from Andrew Ng. Some of you may know him—Andrew is a professor of robotics and machine learning at Stanford University. Maybe some of you know his famous MOOC course on Coursera on machine learning. So if you don't trust me, trust him." The basic thing: what you see here is the amount of data (the amount of training data) and the performance you can gain on a test set the algorithm has not seen before.

The Scaling Problem

What you can see is that with a rising amount of data, most learning algorithms are saturating with respect to performance. This means even if you feed in more data into the training phase, you will have no benefit with respect to performance. In contrast, deep learning scales with the data. As more data, as better the models. "And in the era of big data—we all talk about big data—this is the key. Deep learning is the key because it scales with the data. This is very, very important information."

Neural Network Basics: 1950s Origins

Let's go a little bit more into detail. Basically, deep learning or the basis of deep learning is a neural network which, if you also take the perceptron into account, was born in the 1950s—approximately 60 to 70 years old. What happens in a neural network? You have, let's say, an image. You calculate a set of features—for example, the HOG features we've seen before, any type of features—and this is your input layer. You have additionally a nonlinear layer which maps the result to an output layer.

How Neural Networks Work

What happens here: each neuron is connected with all the neurons in the hidden layer, and these kinds of weights are simply called neuron weights. They have a value between zero and one. Right at the beginning, you initialize these weights randomly, and with training, these weights become changed and learn the structure in the data. "Okay, so far by the book." You additionally have a bias term here, but let's skip it for now.

Adding More Hidden Layers

What is the basic idea of deep learning? The first basic idea is simply to add more hidden layers. Why? Because a hidden layer is a nonlinear transformation, so it makes definitely sense to have more than one nonlinear transformation. But what happens: you take the raw data and from stage to stage you transform it in a nonlinear way. The result of this layer is a nonlinear transformation of this layer, and this means a more complex representation.

Hierarchical Feature Learning

You can see here: you feed in an image. In the first layers, you learn some basic kind of features—let's say wavelet features, gradient features. In the mid-level features in face detection, you, for example, learn parts of a face—let's say eye, a nose, a mouth, or ear, or something like that. In the high-level hierarchies, you learn complete faces. "And this is basically the idea: you transform raw data from basic information into very, very high abstract representations."

Training Deep Networks: The Problem

How do we train this kind of networks? This is the basic thing. We have to train these networks, and the basic idea or the basic algorithm which is used in neural networks for that is backpropagation. What happens here in training: you start with a randomly initialized network and you feed in training examples. You propagate forward through the network. You know that the defect is a one and you expect a one, but basically the network says 0.8, so you make an error of 0.2.

The Vanishing Gradient Problem

What happens now in the basic backpropagation algorithm: you take this error and propagate it back. "Maybe most of you know this kind of algorithm—it's called backpropagation of error." But in a deep network, this doesn't work. Why? Because of the depth of the network, the gradient became smaller and smaller and smaller, and the network is not able to learn an appropriate representation of the features. This means you have no benefit by adding more layers. This was the problem until 2006—it doesn't make sense to add more layers because the network was not able to learn.

The Problem Formalized

The problem, as mentioned before, is the vanishing gradient problem. When you go back, the gradient vanishes. Now the question is: why is deep learning now so popular? Why does it work right now? It works because of two very simple ideas, and the speaker wants to introduce these ideas.

Solution 1: Stacked Autoencoders (2006)

The first idea, introduced by Geoffrey Hinton: instead of learning a complete network, we split the network. This means we take this layer, the first layer, the first two neurons, and simply map the input layer—or take the input layer also as an output layer. This is called the autoencoder. This is a simple autoencoder. Instead of learning the complete network at once, we split the network, and then we're able to learn a very, very small network in fractions. Then we split these neurons, then the other neurons, and so on, and we stack it finally together. Then we have a very, very good pre-initialized network who basically knows what you have to learn, and then we can use fine-tuning on the complete network.

Hinton's 2006 Nature Paper

This is the basic idea of the 2006 paper of Hinton which was published in Nature—it was somehow the starting point of deep learning. It was one of the first times that people were able to train networks with millions of neurons.

Current Training Methods

Today, we don't use this kind of stacked autoencoder training anymore. "I think there will also be a renaissance in this because we can train this network with unlabeled data, and in real-world processes, labeled data are very expensive. So you can train this network with unlabeled data—this is a very, very powerful thing." Today, we basically use supervised data—we have a training set and the label. Instead of a stacked autoencoder, we use techniques from the '80s called dropout. This means we're now able to train a network because we drop out randomly neurons during training.

Dropout Explained

What happens: instead of learning all the neurons, we switch off randomly some of the neurons. Let's say, for example, we take an example, switch off some of the neurons, which is a condensed model, and then go back and do the backpropagation. Dropout is now one of the key techniques. The other technique is regularization (the speaker won't elaborate right now). And activation functions—the activation function is the key to avoid this kind of vanishing. This is the way how deep nets are trained today.

Convolutional Neural Networks for Images

The speaker has only talked about deep neural networks so far, but the application he wants to present is an image processing problem. With fully connected standard neural networks, we end up in a problem. Why? Because when you want to analyze an image in a standard formulation, you have to connect each pixel in an image with each neuron. This is the definition of fully connected—each neuron is connected with each neuron, or here in this case.

The Parameter Explosion Problem

This is a problem because, say, we have an image of 1,000 by 1,000 and we have 200 neurons, which is not so much. We have 200 million neuron weights—200 million parameters in the first layer we have to optimize. This is definitely too much.

Convolutional Solution

So what happens: instead of connecting each pixel to each neuron, we take a group of pixels for one neuron—basically a filter for this convolution. This was the basic idea from Yann LeCun in '89, and this is why it's called deep convolutional because grouping pixels and shifting this block over the image is, as mentioned before, convolution. This is where the network had its name from. And as you can see, we can drastically or dramatically reduce the amount of parameters. Before we had 200 million, now we have hundreds—100 parameters per neuron which we have to optimize.

Three Stages of ConvNets

Some of you may not be familiar with the structure of a deep convolutional net. You have your filters that result in some kind of feature maps. The speaker wants to go a few minutes in detail with respect to this. Each convolution network consists of three stages: the convolution stage, the pooling stage (you don't have to use pooling—this is why it's dashed line), and finally the nonlinear activation. "Nonlinear activation is the key nonlinearity in deep networks."

Stage 1: Convolution

In the convolution stage, you define a set of neurons, and the speaker marks these guys in bold because these are the design hyperparameters. What is a hyperparameter in contrast to parameters? A parameter is a neuron weight—we learn these neuron weights during training. This is a parameter. A design hyperparameter is a parameter we have to define in advance before we start with training the network—for example, the architecture of the network. We have to define the number of neurons, the number of layers is a hyperparameter—we tell the network "you look like this." This is why it's called hyperparameter.

Hyperparameters in Convolution

In the convolution stage, we have, for example, k neurons, n-by-n filter. The number of neurons k and the filter size n-by-n (which we've seen before) we have to define, the step width for example of the pixel—we have to shift this filter over the image. This is all the stuff we have to define in advance. Convolution stage is the first stage.

Stage 2: Pooling

In the second stage, we apply pooling. The pooling stage, as you can see here—the result of the convolution stage is then pooled. You see the information is reduced. How does this happen? Let's say this block here is the result of the convolution stage, and then we simply divide in 2x2 blocks, overlapping by one pixel, and just take the maximum value in a small block. "So I will show it here. As you can see here, we pool 2x2 pixel and the maximum value is seven, so we just take the seven into the next step. So this is basically pooling. It's a very, very, very easy idea."

Stage 3: Activation Functions

In the last step, we have the activation. This kind of feature map is then multiplied with the activation function. Here you can see three different kinds of activation functions. The first one is called the sigmoid function, which is the most traditional one—it's logistic regression. Basically you have a tanh hyperbolic which maps the space into values between one and minus one; here you map it between one and zero. And you have the rectified linear unit (ReLU).

Why ReLU Matters

What is the difference between all these activation functions? The speaker doesn't want to go too much into detail, but one thing is very important: the red one is the actual activation function—this is really the activation. The blue one is the gradient you need for backpropagation. What's very important here: when you see the dotted line of the sigmoid and tanh hyperbolic function, you see that for very, very high values (let's say here) or for very high negative values, you do not learn anything. These values will not have a contribution to the overall result. This means if the activation is very high, you learn nothing because you change nothing.

ReLU Advantage

In contrast, the rectified linear unit for positive values always learns, and this is a key point in deep convnets or in deep learning in general—introducing the rectified linear unit. This is in many cases the method of choice or the activation of choice.

Application: Industrial Defect Detection

Let's go a little bit more into the application. We now have introduced deep convnets and use deep convnets for that. First of all, the speaker wants to show the problem. Here you see different surfaces—this is a publicly available database. "If you want to download it as well, I can share the link with you, no problem." You see different glasses, different kinds of defects. You see that in many cases it's even hard for the human eye to detect the defect. "I will show you the defects here."

The Challenge

The question here was: can we train one single network, one single deep convolutional network, to classify all these different defects in all these different classes with one single network? This is the basic idea. If you think about that, if you have to solve this problem with a standard computer vision algorithm, it is extremely complex because we have different kinds of backgrounds, different kinds of defects, and things like that.

Varying Defect Characteristics

As mentioned before, defects occur with varying size, orientation, shape. A scratch can have any kind of orientation. The question is: how do we apply deep learning to that? "I just want to spend one minute." It's basically a classification problem—most of you know classification. You have a good class, you have a bad class. What you're trying to do is try to find a hyperplane which separates the four classes, and then you will know—then you have this kind of example the network has never seen, you compare with the hyperplane, and then you can say it's good or bad. "We learn from historical data. Supervised learning at its best. So it's pure supervised learning."

Data Preparation Pipeline

Let's go a little bit more into the details. The first thing you have to do is the preparation of data—how do we show the network the data? In this case, in the first step, we do not take the complete images. We sample from the images. This means we extract patches, and we know (because we have the label) that specific patches are in a defect area or in a non-defect area. This is called patch extraction.

Data Augmentation

In the second step, we use standard augmentation algorithms. We shear, we rotate the patches, we mirror the patches, and things like that because then the training set is higher and it's easier for us to deal with rotation and translation in the last step.

Data Shuffling

We fuse everything, and this is very important: if you have a multiclass problem, shuffle the data. If you learn class by class, you will end up in fancy problems because of the gradients. So shuffle the data. It's a standard approach. Since we're able to shuffle the data, when we use this multiplier things, then we feed these extracted blocks into the convolution network and train the network.

Architecture Search: The Key Question

Now, one thing was very important and is still important: what is the best architecture to solve a given problem? This is a key question in deep learning. You have so many hyperparameters. You can change the number of neurons, you can change the number of layers—how many layers do I want? You can change the number of filters and things like that. "And I think it's current research or it's still ongoing research."

Experimental Design

What they've done: they defined the size of the filter (seen in the convolution space or convolution example) of 3x3. "We only use 3x3 filters in every block, in every layer." What they've done was they generated three stages. Each stage had a convolution stage and a max pooling stage followed by a nonlinear activation (ReLU). From depth to depth, we added more hidden layers. Why? Because we want to find out how many hidden layers are necessary to solve our problem, to have the best performance for our problem. This was the reason why they've done this kind of experiment.

Investigating Depth and Width

They added layers from experiment to experiment, and they not only added layers, they also added neurons. This means they not only invested in depth, they also investigated the width. "So how many neurons per layer makes sense? How does the result change when we add more neurons in each layer?"

Results: Depth Investigation

Without going too much into detail, the speaker shows some results. What they investigated here was: how will depth influence the detection results? The depth is the number of layers. How will the number of layers influence the detection results? What you can see here from the baseline to D3—the baseline has, I think, three convolution layers, D3 has nine convolution layers—we only have a performance gain of 1.4%. This means even if you have at least four times more convolution layers, we have just slightly a little bit more performance.

The Parameter Paradox

What is very interesting: in the baseline network, the number of parameters (the number of neuron weights) is approximately 7 million. So 7 million neurons we have to optimize. In D3 (the network with much, much more layers), we only have roughly 4 million neurons. Why does a larger network have fewer parameters? Because the position in the network where the most parameters occur is exactly where the convolution ends and the neural network decision function begins. Because in the baseline layer, we end up with a much higher-dimensional feature set compared to the D3 set, we have much more parameters in the baseline layer.

Interpretation: Problem Complexity

One thing important: "What does this tell us? My suggestion—and maybe you have a different one—is why does it not make sense to add so many layers? Because I think in defect detection we have one problem compared, for example, to people detection: the problem is in many cases not complex enough." This is a scratch. What is a neural network doing with all this data? It transforms it from low-level features to very high-level features. But the object we want to describe is a scratch, and you cannot transform this very, very easy geometry into hundreds of layers. It will simply vanish. So the problem is in some cases not complex enough.

Results: Width Investigation

You can also investigate the width—the number of neurons in each layer. And here it is a little bit different. By adding more neurons, we can gain plus 5% in performance. And this is a huge step. Nevertheless, if you compare all these different kinds of networks, the best network basically is this network, or "I would choose maybe this network because it has the biggest performance." But if you also take the runtime into account, maybe the network with the highest depth and the highest width will win at the end.

Benchmark Comparison

They also benchmarked and compared their work (this one here) to existing work. In red, you see where their network is definitely the best. In black—the black here, for example, is not red because other algorithms are also able to solve this class at 100%. The basic information here: this network has no idea that it is used for defect detection. It only learns from raw data. That's all. The network has no clue what it's doing. These algorithms were specially or only created for defect detection.

The Beauty of Deep Learning

"And this is, from my point of view, the beauty. The deep net only learns from data. That's the only thing the network knows. The other stuff was handcoded, engineered, and things like that—and the network won. And this is a very cool thing." Here you see some examples—examples the network had not seen before. The speaker marks them with red. "You see it is very reliable, as you also see in the numbers. These are just some visual information."

Conclusion: The Next Level

Let the speaker finally conclude. What we learn—or what he wants to say—is deep learning pushes machine learning to the next level. Why? Because in old-style machine learning, we care about the raw data. We shift pixels around. We shift basic information around. What deep learning is doing in contrast: in deep learning we only create architectures and we don't care about the low-level features. You don't care about the pixel value. You only care about the architecture. Find an appropriate architecture to solve a given problem. And this is, from my point of view, the next level in machine learning.

When to Choose Deep Learning

One thing is very important: when to choose deep learning? First of all, you need enough training data. It's not so important if you have labels or not—at some point you need a label, but you need enough data. The next thing: you need massively parallel hardware power. "I'm a fan of Nvidia. I highly recommend Nvidia." You need Nvidia GPUs to do deep learning. There are a lot of frameworks around—the speaker doesn't want to recommend one of them. "TensorFlow is a famous one for 2D. I don't have really a recommendation. It depends."

Problem Characteristics for Deep Learning

How does the problem have to look like that it is appropriate with deep learning? Basically, why is deep learning so important for image processing? Because images can be processed in a hierarchical way. For example, if you do face detection (to remember the example), you first of all start with some low-level pixel features, and then on top you learn nose, ears, and things like that, and finally you learn complete faces. This is the hierarchical trend of features. If you have a problem like this, deep learning is perfect.

When Manual Features Saturate

It also makes sense to use deep learning when the manual feature representation is saturated somehow—"you do not gain more performance with adding more features, then also add deep learning and see what happens."

The Most Important Picture

"And from my point of view, this is the most important picture of that talk: deep learning scales with the data. And this is, from my point of view, extremely important."

Closing and Hiring

"So if you have any questions, I'll try my very best to answer them. Let me just add one comment: if you are searching for a job, if you are searching for a PhD, if you are searching for a student job or whatever, we highly like help, and feel free to send me a CV or whatever you want to my email address. Thank you very much for your attention."

Key Technical Insights and Contributions

This presentation delivers several crucial lessons for applying deep learning to industrial problems. First, the hierarchical abstraction principle explains why deep learning works—progressive nonlinear transformations convert raw pixels into increasingly abstract representations until reaching semantic concepts. This mirrors how humans perceive images: edges → parts → objects → scenes.

Second, the scaling law provides the fundamental economic argument for deep learning: traditional ML saturates with more data, while deep learning continues improving. In the big data era, this makes deep learning not just better but essential—the only approach that can fully utilize massive datasets to achieve superior performance.

Third, problem complexity matching reveals that deeper isn't always better. For simple geometric patterns like scratches, shallow-but-wide networks outperform deep-but-narrow ones because the problem doesn't require extensive hierarchical abstraction. This challenges the "deeper is better" dogma, suggesting architecture should match problem complexity rather than blindly maximizing depth.

Fourth, the end-to-end learning paradigm shift moves machine learning from low-level pixel manipulation to high-level architecture design. Instead of engineering features (HOG, SIFT, wavelets), practitioners now engineer architectures (depth, width, filter sizes). This raises the abstraction level, making ML more about system design than signal processing.

Fifth, convolutional architecture efficiency solves the parameter explosion problem that makes fully-connected networks intractable for images. By using local receptive fields (filters) and weight sharing, ConvNets reduce parameters from 200 million to hundreds while preserving or improving performance—a thousand-fold efficiency gain enabling practical image analysis.

Sixth, the vanishing gradient problem and solutions explain the historical barrier and modern breakthroughs. Gradients dying during backpropagation prevented deep networks until Hinton's 2006 stacked autoencoder pre-training and modern techniques (dropout, ReLU activations, regularization) enabled reliable training of networks with millions of parameters.

Seventh, data preparation methodology matters enormously: patch extraction from labeled regions, augmentation through rotation/mirroring/shearing, and critical shuffling for multiclass problems. The shuffling insight—that class-by-class training causes gradient problems—represents hard-won practical knowledge not obvious from theory.

Eighth, the hyperparameter search problem remains unsolved: finding optimal architecture (layers, neurons, filter sizes) for a given problem requires extensive experimentation. The speaker's systematic approach (varying depth and width independently) provides methodology but no general solution—"still ongoing research."

Finally, the benchmark revelation that their general-purpose network (trained only on raw data with no defect-specific knowledge) beats specialized handcrafted algorithms validates the deep learning paradigm. This mirrors the broader pattern across AI: learned representations outperform engineered ones when sufficient data and compute are available, even when the learned system has no domain knowledge. The network "has no clue what it's doing" yet wins—a profound statement about the power of representation learning over explicit programming.