Call it a gut feeling. I do not think we are using technology to the fullest capacity. As a matter of fact, like a lot of other things in life, I think we are quite wasteful with it and we have caused some level of irreversible harm. That is a really ambiguous statement and I do not have much proof. Again, call it a gut feeling. However, I think there is a lot of science and technology we have not even begun to look at yet. We probably do not know how much we are missing. Do not get me started on the untapped areas of data compression I think we have not considered. There is in one particular area I am looking at right now that I think might work and nobody is considering. I digress.
Add to this that I really have a hard time grasping how machine learning, neural networks and general AI are supposed to work. Most of it is just a lack of time reading about them. A smaller aspect is that I believe these are fads and much too overhyped. They are probably useful tools but not capable of doing the things many claim.
So I decided I wanted to start at the basics. How do neurons in your brain work? We use the word "neural" in machine learning stuff so there has to be some similarity, right? Many other things have been built based on our understanding of natural processes. It makes sense we would want to mimic how our brain is capable of processing vast swathes of information in seconds (or less) and retaining it.
In order to get a good grasp on the goal, I promised I would avoid as many rabbit holes as possible. However, that quickly proved difficult.
To begin with, what is my personal (albeit limited) understanding of what neurons are and/or how the brain works? My personal understanding is that neurons fire electrical signals through connections to other neurons. The connections are called synapses. Somehow, someway, these neurons work together and the signals correspond to processing, memory, controlling muscles and more. I do not know if that understanding is accurate. It is merely my own current understanding.
Ok, so let us dig in and see what we can find. Google and DuckDuckGo are my go-to places to research and I want as many scholarly sources as possible. When I read science or medical journals I agree with myself to take it as slowly as possible. I want to understand every word in the article. Sometimes this means some stuff I read takes days to get through a few pages. It is very much worth it.
The first thing I looked up is: "how neurons work" which first leads me to this page. I immediately realized that my understanding was far too limited and probably vastly inaccurate. One of the first things I learned from this page is that neurons work not just on electrical impulses alone but also on chemical signals. Immediately, some words jumped out at me that I did not recognize.
So I google "what is a glia" which leads me to a few results but I chose this wikipedia entry as my starting point. I did not look up every word and term on this page but I did take some time to read it slowly until I got the idea. Glia helps to keep neurons happy and healthy, in as few words as I can possibly muster. They do garbage cleanup too such as removing dead neurons. They come in different shapes and sizes with different functions. Alright, maybe this is enough to move along to the next thing.
Following the first link further, neurons have "a cell body and two extensions called an axon and a dendrite". Alright, so now I need to lookup these things. I sort of understand what a "cell body" so I will skip that for now and return to it later.
Looking up "what is an axon" leads me to this page explaining pretty well what an axon is all about.
If I had to summarize what an axon is I would say it is the connection that outputs electrical impulses from the neuron to other attached neurons. Now, there are a few other things attached to this axon, which is kind of like an information pipe, but I believe I understand the general idea for the time being so I will come back to the attached components later.
This is starting to make some sense now. Looking up "what is a dendrite" leads me to this wikipedia entry which has a lot of information on a dendrite. I had to read this page a few times and look up some of the terms.
If I had to summarize what a dendrite is I would say it is the receiving end of a neuron that manages the signas it receives, where they should go (if anywhere) and are attached to axons of other neurons. This is starting to resemble a "doubly linked list" in programming terms, if I had to compare these connections to anything at this point.
It is here I learned about action potentials. There are some chemical activities that I do not fully understand but I do understand that the strength of certain impulses, combined together, are part of a total sum of strengths called action potential. Depending on a particular threshold, the action potential turns into action and that action is how a neuron understands to fire an impulse or not.
Now, having said that, I also skipped a few things in the material I read so my understanding up to this point is still subject to inaccuracy but the general ideas actually make a lot of sense.
At this point, looking at images of neurons, they bear a striking resemblance to trees with roots and branches. I fell down a rabbit hole there, looking for connections, for about 30 minutes. I slapped myself and came back only to see another connection.
They also bear a resemblance to how we see space in photographs and telescopes, or even looking at the night sky. It is just a zoomed *out* view of neuron connections (the stars and galaxies). There goes another rabbit hole. Slap myself back.
Ok so I was getting a bit impatient and wanted to understand where the processing power happens. Where is the CPU or GPU on this thing?
Looking up "where do neurons process information" led me to this page describing the general structure of a neuron. I realized there are more intricacies that I did not know about.
Again, trying to summarize what I learned, the dendrites receive chemical information which they turn into electical information. All that information is funneled intothe cell body and, I think, the nucleus. If the action potential is enough then it turns into action. That action is sent through the axon to other neurons.
That still does not explain *how* or *where* information is processed. Where do things like calculations take place? Logic gates? Anything?
The page explains that the "soma" (based on an ancient Greek word meaning "body" which is familiar to me as I speak Greek) is there to keep the neuron functional. Ok so that probably means processing does not happen there.
Reading the rest of the page does not explain how information is processed.
Then I decided to look up "neuron logic gates" and just see what comes up with. That led me to this article on dendrites which explains that dendrites actually have logic/function of their own before electrical impulses are further processed down the road.
Well, if dendrites are responsible for at least *some* logic or functional power, then how does it work? Exactly? The article very briefly touches on it and the answer is, more or less, we do not know.
This is a blocker. There is some data to correlate the length of dendrites with their ability to process information. There is also some data to correlate rhythms or clocks in your brain as to how well you can process and recall information.
There is so much more to dig into here. I will have to follow up with more articles to see how far I can go.
I am not entirely sure. There is a part of me that wants to say yes because I really want to say yes. The truth is, I think that (a) we do not know enough about how neurons work together to understand how we might even begin to simulate it in code and (b) I personally do not know enough yet one way or another.
If I were forced to come up with some theory right this moment, I would say that neurons do not function in isolation. Instead, since there are several types of neurons with differing functions, I would say our brain processes information in tandem with many different types of neurons at once. Where neurons are located, how they are triggered, how they are connected together, all these things seem to have a role.
One thing I believe is really clear is that the individual pieces of our brains neural network are very well suited to deconstructing large pieces of information into much *much* smaller pieces and relating them together. For example, the color red is a basic component of many facets of our lives. Our brains can take the color red and relate them to things like lipstick, a car we like, clothing, shoes, whatever. We can recall these far more complicated objects to mind just by thinking of the color red. My current understanding of how neurons work seem to clearly indicate that our brains neural networks function like this.
This kind of deconstruction seems very well suited to something else. Memorization and neural plasticity.
These are all topics for other parts of this series. However, putting this together into code would seem to be something like building basic "lego" blocks of functions that are very dependent on context and specific inputs to do their jobs. At a higher level, you would process information by breaking it down into core components, processing them through the "lego" functions and then memorizing the connections and output at the end. Further information would be broken down again but then you could determine portions of the information you already recognize and begin building relationships between pieces of information. These relationships are the easiest to remember and retain (like the color red) but then allow you to access "cold storage" information like "I remember this lipstick was red. Oh and this car was too! Oh and..." etc.
I have gone down a pretty deep rabbit hole. I will leave it here and continue with part 2 soon.