Note: Please note that this isn’t meant to be a valid neuroscience or research, rather it is my somewhat crude attempt to understand how the brain functions from my perspective, and try to create a model that mathematically falsifiable to test that understanding.

TLDR; What happens to a system’s output when its input encounters a non-ideal environment? Here, I turn part of my engineering analogy into an exploratory experiment: perturb an image with additive noise, multiplicative noise, and Gaussian blur, then measure how its representation and predicted label change. Additive noise increased the embedding’s distance from its baseline while often leaving the winning label unchanged. Under multiplicative noise, that distance quickly approached a plateau. Blur led to a second question: can a cat’s representation approach a car’s? The strongest approach occurred when both images were blurred, but their representations remained separated and the separate classifier still gave them different labels. These are measurements of pretrained models, not a demonstrated explanation of a human language mistake.

Introduction

In December 2022, I wrote an article titled “An Engineer’s Point of View on the Brain” where I tried to understand and conceptualize cognitive processes, information storage and retrieval, ideal and non-ideal states, and more using computational models. I concluded the article on the note that the brain is an information dependent undecipherable black box that is faster than we’d like to admit; with two model. A trie which speaks to the interconnectedness of the brain, and the disk, RAM and cache model which speaks to how the brain might store and retrieve information.

These models were an attempt to find an engineering analogue for the human brain. The earlier article left me with a question rather than a tested explanation. Here, I want to make part of that question measurable: how do representations and predictions change when we perturb an input? The experiment can test specific expectations about these models; it can’t, by itself, validate the analogy to the brain.

Motivation

The human brain is a compact biological computer that has inspired research & technological advancements alike, but regardless, it is so easy for the brain to be prone to errors, we can all agree that “I’d like to park my cat” makes no sense, but for some weird reason, it slipped out when you intended to say “I’d like to park my car”. The Medium story I wrote tried to investigate this by proposing models, mainly aimed at understanding where the information storage and retrieval process went wrong that led to the unwanted sequence of words.

The post builds upon those models, but gives me a smaller task to work with: define the transformations, measure their effects, and see which parts of my original intuition survive the experiment.

Review

Unlike the “Disk, RAM, and Cache” model, the “Huge Trie-List” model introduces something quite interesting, an ideal and non-ideal (noisy) state. The ideal state says the brain is stimuli dependent function \(f(X)\), and for every set of stimuli \((X)\) there’s a corresponding set of outputs \((Z)\), and I defined it as;

This is fine if we treat the brain like a pure mathematical function; but as established in the earlier post, just like any biological, mechanical, electronic, or electro-mechanical system, there is “noise” (internal & external noise). What happens when we take them into account; \(X, X{\phi}, X{\rho}\); \(X{\phi} = \text{internal noise}, X{\rho} = \text{external noise}\); \(f(X)\) is now given by;

My starting intuition was that unwanted stimuli could still affect a system’s output. That is the part I want to explore here. It doesn’t establish that the brain processes every stimulus alike, or that it never filters noise.

Experiment Setup

For the purpose of this experiment and ease of visualization of the effects of the modelling, I’ll be using images with dimensions, 512 by 512 by 4, i.e. a 512 by 512 image with 4-channels (RGBA), that will be converted to 3-channels (RGB).

We Need a Brain

To test this, we need a stand-in for the brain. Don’t worry, there’s no Frankenstein stuff going on here; I’ll use pretrained neural networks. I take two separate readouts: an image embedding for the learned representation, and a predicted label. These come from different models, so let’s keep that distinction in mind: a change in one model’s embedding doesn’t tell us where the other model’s decision boundary lies.

Learned Representation

The learned representation is the first function, \(y = f(x)\), that takes in our image \((x)\) and return an embedding which is the model’s representation of the image being fed to \(f(x)\). To extract the learned representation I utilized facebook/dinov2-base from HuggingFace. This function is given by;

To get the desired learned representation I selected the [CLS] token from the embedding, which is a global representation of the entire image \((x)\), 1 CLS token + 256 patch tokens. The learned representation function is now given by;

\(Y\) is a 768-dimensional vector which is the model’s global representation of the image. Further down in the experimentation, this representation will be used to perform these calculations.

Label

The facebook/dinov2-base doesn’t have a classification head to predict a given label, so I employed the use of facebook/dinov2-giant-imagenet1k-1-layer which has an ImageNet classification head with \(1,000\) class labels, \(C(x)\). Applying “\(\operatorname{argmax}\)” to the output logits of the classification head’s process from \(f_{\text{DINOv2-Giant}}(x)\) gives us the label of the image \((x)\).

\(C\) is the predicted class from the given image \((x)\), this is a semantic classification readout in addition to the embedding from facebook/dinov2-base.

Establishing the Baselines

Before proceeding, it is good scientific practice that baselines are established for all experimentation that’ll be performed. I used an image of a cat for this experiment and cropped it to the aforementioned dimensions i.e. 512 by 512.

Fig 1. The cropped image of cat image gotten from Dear Sunflower on Unsplash, hosted on Github.

Now, we establish the baseline learned representation \((Y_{0})\) and the baseline semantic classification readout \((C_{0})\) for our input image of a cat \((x)\), these are given by;

After running this baseline experiment we get the following baseline values;

Introducing Noise (Perturbation)

Now that we have established our baseline (ideal condition), we’d like to introduce the “noise”, the question now is “How much can we perturb the input image \(X\), that the learned representation \(Y\) changes?”. Now this shift our function from \(y = f(x)\) into a function that now takes into cognizanse, the noise, \(N\). This is now given by;

Notice the difference? \(X\phi\) and \(X\rho\) are now collapsed into \(N\). But we don’t only want to apply \(N\), we also want to apply \(N\) at varying degrees of magnitude, and hence we introduce the “perturbation magnitude” \((\delta)\). We apply fixed gaussian noise \((N)\) at varying perturbation magnitudes \((\delta_{i})\) for \(i = \lbrace1, 2, 3,…, 255\rbrace\), this function is now given by;

Because each pixel values \((p)\) are represented as \(p \in [0, 255]\), we have to make sure we don’t overshoot that threshold, so we clip the perturbed values between 0 and 255, this is given by;

To see how much \(Y_{i}\) changes with increase in perturbation magnitude \((\delta_{i})\) on our fixed gaussian noise \((N)\). We measure the euclidean distance \((D)\) between the baseline learned representation \((Y_{0})\) and the varying learned representations \((Y_{i})\), this is given by;

In addition to this, we want to see how the distance from the baseline changes as \(\delta\) increases by \(1\). I use \(\Delta D\) for this successive change in \(D\), not for the distance between adjacent embeddings. This is given by;

Now we run the pipeline with a fixed noise realization and increasing perturbation magnitude. The distance from the baseline rises, but the size of each successive increase changes along the way.

The distance rises quickly at first, grows more slowly around \(\delta=100\), then continues upward before flattening towards the end of the sweep. The successive-change plot makes the changing rate easier to see: \(\Delta D\) is initially large, then becomes smaller, with fluctuations. Near the end it is mostly below \(0.2\). This is a change in distance from the baseline, not a measurement of the angle between embeddings.

So, as perturbation magnitude becomes increasingly higher the incremental change in this distance becomes smaller. Looking at the semantic classification \((C_{i})\), and this is the interesting part; \(C_{i}\) remained \(C_{i} = \text{“Egyptian cat”}\) through \(i \in \lbrace1, 2, 3,…, 255\rbrace\). Regardless of increasing perturbation magnitude the semantic classification readout still remained \(\text{“Egyptian cat”}\). To peer into the details, we looked at the behavior of the class probability and we see the actual behavior of the semantic readout.

Table 1. The classification probabilities with increasing perturbation magnitude while still predicting \(\text{“Egyptian cat”}\).

While \(\text{“Egyptian cat”}\) remained the highest probability, it also fluctuated substantially, demonstrating that stable semantic classification did not imply a stable classifier confidence, showing classification invariance without probability invariance.

I repeated the experiment with different noise realizations. On the eighth run, the classifier’s winning label changed at \(\delta=230\). The base-model embedding also moved under perturbation, but that distance doesn’t locate the separate classifier’s decision boundary.

This is exactly the kind of behavior we wanted. At \(\delta = 229\) the probability for \(\text{“Egyptian cat”}\) was \(29.0702\%\), and at \(\delta = 230\) the probability for \(\text{“tiger cat”}\) was \(29.3101\%\) and steadily increased to \(37.3848\%\) at \(\delta = 255\). With the noise realization fixed, increasing \(\delta\) scales that noise before clipping; it doesn’t establish that the embedding keeps the same direction. To vary the noise realization as well, we use;

Now, what happens if we change the noise realization at each step too? We would be changing its direction as well as its magnitude, before clipping. That gives us a different experiment from following one fixed noise direction.

What we notice is that changing the direction of the noise realization by increasing values of \(\delta\) still maintains the upwards trends with fluctuation that become more consequential with increase in perturbation magnitude.

Make it Multiplicative

So far, we have added noise to the image. What happens if we multiply the pixel values by noise-dependent factors instead? This makes the perturbation depend on the input’s pixel values. With the noise field fixed, multiplication is linear in the input before clipping; clipping is the nonlinear step.

We perturb the image multiplicatively, given by the equation above aiming to shift the representation in the representation space by a significant amount.

Okay. The distance from the baseline rises sharply, then settles around a plateau with small fluctuations. Its successive change, \(\Delta D\), fluctuates around zero after the initial jump. The classifier also moves away from \(\text{“Egyptian cat”}\) almost immediately. The plateau tells us that this distance changes little at higher magnitudes; it doesn’t mean the embeddings stop changing, or establish why the plateau occurs.

Something jumps out immediately from our experiments above, we have been treating noise as “noise” from a signal processing perspective, while this is valid, there are many ways of perturbing the input \((x)\) and additive noise is one of them, but maybe not the best for our particular experimentation.

Blur, blur, blur sheep

The noise experiments show that representations can move substantially while the original label sometimes remains unchanged. They don’t tell us how much distinguishing information is left in the image. Now, enters Gaussian blur: what happens when we progressively suppress fine spatial detail instead? A new transformation function emerges, given by;

The new transformation function \((T)\) performs a transformation on input \((x)\) using Gaussian blur \((B)\) and with blur intensity \((\sigma)\), this function progressively removes information from the input image.

Under blur, the winning label changes early and sometimes returns to an earlier class. The image is classified as \(\text{“Egyptian cat”}\) at \(\sigma=1,2,3\), then as \(\text{“Japanese spaniel”}\) at \(\sigma=4\) and \(\text{“Blenheim spaniel”}\) at \(\sigma=6\). It returns to Egyptian cat at \(\sigma=7,8\), before changing again. By \(\sigma=255\), the prediction is \(\text{“digital clock”}\). The table below follows both the winning label and the probability assigned to the original class; those are different quantities.

Table 2. Selected classifications with increasing blur intensity, highlighting the return to Egyptian cat at \(\sigma=7\) (also predicted at \(\sigma=8\)).

Where do we meet?

Before we go further, there have been experiments similar to what we’re doing here. Dodge & Karam compared how humans and deep neural networks recognize images under distortions like blur and noise; the tested networks performed worse than humans, and their errors had little correlation with human errors.¹ Geirhos et al. explored this across multiple image degradations, and saw human and network error patterns increasingly diverge as the signal weakened.²

Blur itself has been investigated too. Vasiljevic, Chakrabarti & Shakhnarovich showed that networks trained on clear images lose recognition accuracy under blur, but fine-tuning with blurred images can recover much of that accuracy and produce blur-invariant representations.³ More recently, Jang & Tong found that training with clear and blurred images improved how well CNNs predicted measured neural responses and aspects of human recognition behavior.⁴ So, the way a network responds to blur also depends on what it was trained on.

What about the internal representation? Yoshihara, Fukiage & Nishida looked at how blur training changes those representations, comparing responses to sharp and blurred images from the same image, different images within the same class, and images from different classes.⁵ Comparing representations under blur therefore already has a body of work behind it.

My question comes from the earlier Medium post; I wanted a way to test part of what I said there with something we can define and measure. For this experiment I keep the pretrained models fixed, perturb the input, and follow the changes in learned representation and semantic readout. This gives us a way to test that part of the engineering analogy, but it doesn’t establish a model of what happens in a human brain.

How close can we get?

With the baselines established, we saw how additive & multiplicative noise, and then blur, moved the cat’s learned representation away from its original representation. We also saw the separate classifier change its prediction. Now, let’s take this further; how close can the representation of a car get to that of a cat?

The idea is to blur both images and measure the distance between their representations, then repeat the comparison while keeping one image unblurred. If the representations do not come close to each other, that there’s no convergence for this image pair and blur sweep. But, if they converge, we would still need to see whether they approach the other image’s original representation.

To get started I introduced a second image, a car classified as \(\text{“minivan”}\), and applied the same blur strength to both images. The transformed images are \(T_{cat,\sigma}=B_\sigma(x_{cat})\) and \(T_{car,\sigma}=B_\sigma(x_{car})\); passing them through our representation model gives us \(Y_{cat,\sigma}\) and \(Y_{car,\sigma}\). This is given by;

\(B_0(x)=x\) is the unblurred baseline, with representations \(Y_{cat,0}\) and \(Y_{car,0}\). Remember, we are using a 768-dimensional representation for these comparisons, recording 256 embeddings for each image; the baseline and blur strengths from \(\sigma=1\) to \(\sigma=255\).

Blurring both images

We measure the Euclidean distance between \(Y_{cat,\sigma}\) and \(Y_{car,\sigma}\) as we progressively increase blur at the same varying strengths, then observe how close or how far apart their representations are in the representation space.

The baseline distance of \(66.8234\) falls to \(59.4186\) at \(\sigma=4\), and \(58.1713\) at \(\sigma=8\) as we see the representations getting closer, but the distance doesn’t keep falling at every step; at \(\sigma=15\), it rises to \(62.9037\). It then decreases with fluctuations and then settles around \(36.3616\) at \(\sigma=167\), and ending at \(40.3695\) at \(\sigma=255\).

At their closest, the distance between them was \(45.6\% ~ \textbf{lower}\) than the baseline distance. Does this mean the cat’s representation is becoming like the original car’s representation? No; we are blurring both images, they get closer as the information distinguishing them is lost, without ever approaching the other image’s original representation infact at the highest blur the distance rises again.

Holding one image fixed

What we observed from \({D}_1\) is resulted from both representations moving. Now let’s remove that movement from one image. Let’s introduce two more measurements alongside \({D}_1\); \({D}_2\) compares the blurred cat with the original car, while \({D}_3\) compares the original cat with the blurred car. This gives us three pairwise distance measure given by;

At \(\sigma=0\), all three distance measurements compare the same original pair, so they start at the same distance. Let’s sample out the point at which the two blurred representations get closer and where it settles, \(\sigma=167\), and the end of the sweep, \(\sigma=255\);

Table 3. Pairwise Euclidean distances before blur, at the closest recorded point for the two blurred representations, and at the final blur strength.

The observation above so that at \(\sigma=167\), the distance between the two blurred representations has falls to \(36.3616\) and settles at \(36.3616\), while the other distance measures are \(65.2932\) and \(64.7817\). At \(\sigma=255\) those \({D}_2\), \({D}_3\) distances measures are \(64.6428\) and \(66.4435\), both do not show significant shift from baseline distance of \(66.8234\).

Fig 10. Euclidean distance of pairs \(D_{1}\), \(D_{2}\), and \(D_{3}\) against the increasing blur intensity \((\sigma)\).

Looking at the plot, the two blurred representations get closer to each other much more than they approach the other image’s original representation. This retains the idea that losing their distinguishing information brings them closer, but as established earlier the distances alone doesn’t tell us much. While Euclidean distance depends on the magnitudes of the vector embedding (representation). What happens to their direction?

Are they pointing in the same direction?

For this we use a common spatial distance measure, “cosine similarity”, using the same three representation pairs from above, but instead of measuring the distance between them, we now measure their directions in the representation space. For nonzero vectors \(u\) and \(v\) from our pair, this is given by;

A value closer to \(1.0\) means the vectors are more aligned or move in the same direction. We use \(\mathrm{cos}D_1\), \(\mathrm{cos}D_2\), and \(\mathrm{cos}D_3\) from the same pairs \({D}_1\), \({D}_2\), and \({D}_3\). These are names for the pairwise cosine similarities; we calculate them from the embeddings, not cosine of the distances.

Table 4. Cosine similarities for the same three 768-dimensional representation pairs, including the strongest recorded alignment of the two blurred images at \(\sigma=167\).

Again, we see the two blurred representations coming closer. \(\mathrm{cos}D_1\) starts at \(0.0180\) without blur and reaches a high similarity score of \(0.6874\) at \(\sigma=167\) where we observed the closest distance. The score then fluctuates, settling at \(0.6216\) at \(\sigma=255\), showing us that increasing the blur strength doesn’t keep increasing cosine similarity.

Now let’s look at \(\mathrm{cos}D_2\) and \(\mathrm{cos}D_3\). \(\mathrm{cos}D_2\) only reaches a score of \(0.0771\), while \(\mathrm{cos}D_3\) fluctuates approximately around \(-0.0432\) to \(0.0322\). At \(\sigma=167\), their scores are \(0.0444\) and \(0.0264\), respectively. They stay closer to zero, showing little to no similarity with other image’s original representation. Seeing the slightly negative values in \(\mathrm{cos}D_3\) means the similarity stay near zero; not negative alignment in the opposite direction.

Fig 11. Cosine similarity score of pairs \(\mathrm{cos}D_{1}\), \(\mathrm{cos}D_{2}\), and \(\mathrm{cos}D_{3}\) against the increasing blur intensity \((\sigma)\).

We see cosine similarity of \({D}_1\) increase with the directions the blurred embeddings becoming more aligned which showed partial convergence between the two blurred representations with a substantial distance still remaining. Again, this doesn’t mean that cat and car are becoming the same.

At the similarity score, at \(\sigma=167\), the cat image was classified as \(\text{“wire-haired fox terrier”}\), while the car image was classified as \(\text{“television, television system”}\), with probabilities averaging \(7.9\%\) and \(8.0\%\) respectively. At \(\sigma=255\), we get \(\text{“digital clock”}\) and \(\text{“microphone, mike”}\). Even at the point when they are closest, the separate classifier doesn’t give them the same label or anything similar.

Where this leaves us?

The question that started this was simple; how do we end up saying “I’d like to park my cat” when we meant “car”? By introducing noise and blur, we saw the model’s representation move and classifier prediction change. Blurring both images also brought their representations closer, but keeping one image unblurred showed that they didn’t get close to the other’s original representation.

The interesting part of this experimentation is how two representations became more alike as the information distinguishing their inputs disappears. Getting closer in representation space also didn’t give us the same label from the classifier. These observations have not addressed why a human would say “cat” when they meant “car”. The truth is there are so many things that can make a human “glitch” in the manner that built this hypothesis; heck, through the course of write this post I made several similar mistakes.

In my case I was exhausted and stressed after a wedding, which is another angle that might be worth exploring; maybe exhaustion or stress could translated to jittering or perturbing the system (model) and not the input.

This experiment gave me a way to measure parts of my idea from the initial post, and showed me where that idea needs more work. For now, I have something concrete to test further, rather than a demonstrated explanation of what happens in the brain; and that is okay.