Before Language On learning, understanding, and the possibility of an artificial mind

We've become so accustomed to communicating with AI through language that we've gradually come to view language as intelligence itself.

But humans don't begin understanding the world by learning language. Before we can describe an apple, we can already see it, touch it, and observe what happens when it falls to the ground.

Language enables us to communicate, organize, and share knowledge, but it is not necessarily the starting point of understanding.

So, is it possible to construct an artificial nervous system that first learns about the world through experience, and then gradually learns language?

I don't know the answer.

Before Words

Consider a child who has never learned the word gravity.

The child may have dropped countless objects and gradually learned to expect them to fall. Years later, a teacher introduces Newton's laws and explains the phenomenon mathematically.

The theory provides a more precise way to describe and reason about something already familiar.

Of course, this early understanding is incomplete, and sometimes incorrect. Still, the child does not need to know an equation to form expectations about the world.

This makes me wonder whether there is another path toward artificial intelligence.

Today's language models are remarkably capable. They can reason, write, translate, and solve problems that would take humans years of education to understand. Multimodal models can also process images, sound, and video.

Their achievements are undeniable.

Yet I wonder whether understanding must begin with symbols and language, or whether a system could acquire knowledge through experience before learning how to describe it.

An object does not come with a label. A falling apple does not announce that it is obeying gravity. The world simply exists, changes, and responds to what happens within it.

Perhaps an intelligent system could begin there.

A Mind That Continues

Most AI systems we interact with follow a familiar pattern: we provide an input, the model processes it, and it generates an output.

I imagine something different.

A system that continues to exist even when nobody is talking to it.

Its internal state changes over time, influenced by its previous experiences, its environment, and its own activity. It may observe something, remember something, or do nothing at all.

Not every input needs a response. Not every internal process needs to become a sentence.

I would like such a system to learn continuously, without a strict separation between training and inference.

One possible starting point is a Spiking Neural Network (SNN).

Biological neurons communicate through electrical events over time. SNNs attempt to model certain aspects of this behavior using artificial neurons that emit spikes.

Combined with recurrent connections and synaptic plasticity, they offer a way to investigate how experience might alter the future behavior of a network.

They also provide an interesting perspective on time. Rather than treating time merely as another input value, the network can possess internal states that evolve and decay.

However, an SNN is not inherently intelligent. Spontaneous activity is not necessarily thought, and changing synaptic weights does not necessarily mean acquiring useful knowledge.

I don't want to assume that reproducing a few properties of biological neurons will somehow produce intelligence.

I only think it might be a reasonable place to begin experimenting.

The real challenge is understanding how experience could give rise to stable memories, reusable representations, and eventually knowledge about the world.

Learning Without a Teacher

Another question is why such a system would learn in the first place.

Humans learn through experience, exploration, imitation, and education. Some learning happens without anyone explicitly teaching us. Other things are learned because someone encourages us, rewards us, or even forces us to study.

But human babies are not blank systems. Evolution has already provided us with senses, reflexes, biological needs, learning mechanisms, and many other useful structures.

An artificial system would not inherit these automatically.

I don't think it is realistic to reproduce the entire process of biological evolution before building a learning system.

Instead, I would like to investigate which minimal mechanisms might be necessary for learning to begin.

Perhaps these would include recurrent activity, short- and long-term memory, local synaptic plasticity, and mechanisms that prevent the network from becoming completely silent or excessively active.

I would prefer not to give it a predefined collection of concepts such as apple, cat, or person. Nor would I want every learning experience to depend on a human assigning a correct answer.

That does not mean rewards and punishments should be forbidden.

External feedback is part of the world. Education can guide a learner, and feedback can make certain experiences more important than others.

But the absence of a reward should not automatically mean punishment, and learning should not become impossible whenever no teacher is present.

I imagine a system that can learn some things simply through experience, while external feedback can influence other aspects of its behavior.

Whether such a combination could support increasingly complex learning remains an open question.

Language as an Acquired Ability

Suppose an artificial system had already developed some understanding of its environment, however limited.

What would language mean to it?

Perhaps it could gradually learn that certain sounds, visual symbols, or communication patterns correspond to things it has experienced.

The word apple could become associated with an already familiar kind of object, rather than being the foundation from which the object is understood.

Language might then become useful not only for communicating with others, but also for organizing and extending existing knowledge.

I also want to distinguish three different things:

• Understanding: internal representations of objects, events, and relationships.

• Language: a system of meaningful expressions and rules.

• Expression: the physical or digital means by which language is communicated.

Humans can speak, write, or type. These are different ways of expressing language, but they do not require us to represent every thought as sound, handwriting, or bytes.

Why should an artificial intelligence be limited to one method?

I imagine allowing it to learn several forms of expression.

It might eventually learn to communicate through sound, through written symbols, and through a digital communication interface.

For example, it could learn to operate a device that accepts UTF-8.

UTF-8 would not be built into its internal understanding. It would be an external communication protocol, just as a keyboard is an external tool for humans.

The system might learn which signals cause the device to produce certain characters, and eventually how to use those characters to express language.

This would be a third means of expression, independent of speaking or writing. There would be no need to simulate vocal cords or human fingers merely to let it communicate digitally.

It is important to distinguish learning a communication protocol from learning language itself. Producing valid UTF-8 does not imply understanding what the text means.

The goal is not to make a neural network generate tokens.

The goal is to investigate whether a system could develop something meaningful to express, and then learn how to express it.

Notes for a Future Attempt

What follows is not a proven architecture. It is a record of the technical ideas I currently have, so that I can return to them without having to reconstruct the entire discussion.

Some decisions are fairly straightforward engineering choices. Others are unresolved research problems.

I want to keep that distinction clear.

1. General Principles

The intended system should have the following properties:

A particularly important distinction is between the system's internal representation and the interfaces through which it interacts with the environment.

A camera produces pixels. A microphone produces sampled signals. A digital port transmits bits or bytes.

These are properties of the interfaces, not necessarily the formats in which the system must represent its knowledge.

Likewise, giving a system an output channel does not mean that the channel determines how it thinks.

2. A Minimal SNN

The first implementation should be small enough to understand and debug.

I would start with a recurrent network of approximately 256 neurons, using a simple Leaky Integrate-and-Fire (LIF) model.

Each neuron maintains a membrane potential that changes in response to incoming synaptic activity and gradually decays toward a resting potential.

A typical formulation is:

τ_m · dV/dt = −(V − V_rest) + R · I(t)

When the membrane potential crosses a threshold, the neuron emits a spike, resets its potential, and enters a refractory period.

The initial simulator could use:

None of these parameters should be treated as biologically optimal.

A fixed time step is a numerical approximation. Biological neurons do not share a global clock, and an event-driven simulator might eventually be more efficient.

For the initial Rust implementation, I would separate neuron state, network connectivity, plasticity, and external input.

Sparse adjacency lists should be sufficient for small experiments. Struct-of-arrays storage may later make simulation and vectorization more efficient.

GPU acceleration is not a priority. Understanding the behavior of a small network is more important than simulating millions of neurons whose activity I cannot explain.

3. How Should It Learn?

The first learning mechanism to investigate is Spike-Timing-Dependent Plasticity (STDP).

In a simplified version, the change in synaptic strength depends on the relative timing of presynaptic and postsynaptic spikes.

Repeated temporal relationships can therefore change the connections between neurons.

This provides a basic mechanism for experience-dependent learning without requiring labeled examples.

However, STDP by itself may be insufficient for stable, generalizable knowledge.

It may also require mechanisms for controlling overall activity, preventing runaway excitation, and avoiding undesirable changes to previously learned connections.

Possible mechanisms to investigate include:

• Synaptic scaling and homeostasis.

• Inhibitory regulation.

• Short-term synaptic dynamics.

• Different time constants for neural and synaptic states.

• Metaplasticity, where the rules governing plasticity can themselves change.

• Neuromodulated plasticity.

For external feedback, one possible approach is a three-factor learning rule.

In addition to presynaptic and postsynaptic activity, a third signal influences whether particular synaptic changes are strengthened or weakened.

A simplified experimental form could be:

Δw_ij = η_local · H_ij + η_mod · m(t) · e_ij

Here:

• H_ij represents a local activity-dependent learning term.

• e_ij is a temporary eligibility trace recording relevant recent activity.

• m(t) is a modulatory signal, potentially influenced by external feedback.

• The two learning rates control the respective contributions.

This is only a candidate learning rule, not a complete biological model.

The separate local term is intentional: the network should retain a mechanism for learning even when no external feedback is available.

I would compare networks with and without modulation instead of assuming that adding rewards necessarily makes them better.

4. Time, Memory, and Spontaneous Activity

Time should be part of the system's dynamics, not merely a number supplied with each input.

Different internal processes could operate at different time scales:

• Milliseconds: spikes and membrane potentials.

• Tens or hundreds of milliseconds: short-term neural traces.

• Seconds or longer: adaptation, recurrent states, and eligibility traces.

• Much longer periods: persistent synaptic changes.

These are illustrative ranges, not rigid boundaries.

The combination might allow the network to respond differently to the same stimulus depending on what happened earlier.

Recurrent connections could also support activity after a stimulus disappears.

However, maintaining spontaneous activity is not the same as sustaining meaningful thought.

One of the first research questions should be whether internal dynamics can remain stable, varied, and sensitive to experience without collapsing into silence, saturation, or meaningless repetition.

A related question is whether previously encountered activity patterns can reappear spontaneously, and whether such replay serves any useful function.

5. Perception and the Environment

I would initially avoid complex real-world inputs.

Before introducing images, the system should be tested with simple temporal signals, such as:

A → B

and later:

A → B → C

It should be possible to change the timing, order, and frequency of these events.

Only after basic temporal learning is measurable would I introduce vision.

A possible first visual environment would consist of low-resolution grayscale images containing a few geometric objects.

The environment could show objects moving, disappearing behind obstacles, and reappearing.

A simple input encoder might convert pixel intensity or intensity changes into neural activity.

This encoder would be an engineering interface, not a pretrained object recognizer. I would avoid providing object labels directly to the network.

The more interesting experiments would ask whether it can develop representations that remain useful when an object moves, changes appearance, or becomes partially hidden.

Text could later be introduced in two different ways:

Visual text: characters appear as images, just as humans see writing on a screen.

Digital input: bytes arrive through a separate communication device.

These channels must not be confused.

A byte stream is not an internal thought process, and recognizing the visual appearance of a word is not necessarily understanding its meaning.

The system should eventually have the opportunity to associate these experiences, but the association itself is something to be learned.

6. Actions and Communication

An artificial nervous system should eventually be able to affect its environment.

However, reproducing the detailed mechanics of human movement would introduce a separate and difficult research problem.

I do not want to require an artificial hand merely to let the system type, or a simulated vocal tract merely to let it communicate.

For early experiments, a small number of abstract action channels should be sufficient.

For example, an environment might expose two actions that produce different observable consequences.

The system would have to discover the relationship between its activity and those consequences.

Later, a separate digital communication device could be introduced.

A possible interface would contain eight data lines and a send signal:

D0 ... D7 + SEND → one byte

The environment would interpret the transmitted bytes according to a protocol such as UTF-8.

The neural system would not be told that a particular neuron means a, b, or any other character.

Instead, it would need to learn how to control the interface and how different outputs affect the environment.

The device could display its output on a screen, making the results observable through vision.

It might also receive digital messages from another intelligent agent.

This creates the possibility of learning correspondences between communication signals and experiences.

A lower-level serial bit interface is also possible, but learning electrical or timing conventions is a different problem from understanding language. I would not introduce that additional difficulty without a reason.

Most importantly, successful communication should be tested through meaning and generalization, not merely by checking whether the system produces valid characters.

7. External Rewards and Internal Motivation

I do not want to design a system whose every action is selected to maximize a human-defined reward.

But I also do not think external feedback should be excluded.

There are at least three distinct situations:

No feedback should not automatically be treated as negative feedback.

The network should remain capable of changing through experience without receiving rewards.

External feedback could later modulate learning, attention, or behavior.

The much harder question is what could make the system explore spontaneously.

Random activity alone is not curiosity. A hand-designed novelty score is not necessarily a general solution either.

Intrinsic motivation, homeostatic regulation, and open-ended learning are existing research directions worth investigating.

I would prefer to test these mechanisms individually, rather than combining them and calling the result intelligence.

8. Experimental Roadmap

If I ever begin implementing this, I should proceed through increasingly difficult experiments.

Each stage should have observable results and proper controls.

These stages are not a guaranteed progression.

In particular, the transition from simple associations to stable concepts, language, and abstract reasoning may require mechanisms that have not yet been identified.

The roadmap is a sequence of questions to investigate, not a promise that sufficiently many neurons and enough time will solve them.

9. How to Evaluate Learning

A network producing complicated activity is not evidence of understanding.

Every experiment should distinguish learned behavior from randomness, transient activity, and memorization.

The first experiments should compare:

• A network before and after exposure to repeated stimuli.

• An otherwise identical network with plasticity disabled.

• Networks trained with different sequences or timing.

• Learned responses to familiar inputs and previously unseen variations.

• Networks with and without external feedback.

Experiments should use fixed random seeds when reproducibility matters, multiple seeds when measuring reliability, and separate learning and evaluation periods.

Important records would include the network configuration, input events, spike timestamps, synaptic weights, and behavioral outcomes.

For later experiments, I would pay particular attention to generalization and intervention.

If a system responds correctly only to the exact examples it has seen, that is not enough.

If it has learned something about object persistence, for instance, it should behave appropriately when an unfamiliar object becomes occluded and later reappears.

If it has learned a causal relationship, changing that relationship should reveal whether it can detect and adapt to the change.

No single test would prove that it truly understands the world. But increasingly demanding tests could help distinguish useful internal models from shallow associations.

10. What I Still Don't Know

There are several questions I do not have good answers to:

1. 1.

   What are the minimum innate structures necessary for open-ended learning?

2. 2.

   Can local synaptic plasticity support long-term, generalizable representations?

3. 3.

   How should memory remain stable without preventing further learning?

4. 4.

   Can spontaneous neural activity become useful internal processing?

5. 5.

   How could an agent discover new behaviors without predefined goals?

6. 6.

   What mechanism would allow it to represent persistent objects and causal relationships?

7. 7.

   How could meanings become associated with symbols through experience?

8. 8.

   How would the system decide when and what to communicate?

9. 9.

   Could language become useful for internal reasoning without becoming the required format of all thought?

10. 10.

    How could an artificial system eventually acquire abstract theories from accumulated experience?

11. 11.

    Is an SNN even the right foundation for these questions?

That last question matters.

I currently prefer SNNs because of their temporal dynamics and biological inspiration. But my actual goal is not to prove that SNNs are superior.

If another architecture turns out to be more suitable, I should be willing to change direction.

11. Research Worth Revisiting

These ideas overlap with several existing fields. I should revisit their literature before attempting to invent everything myself.

Spiking neural networks and learning

• Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules — Frémaux and Gerstner, 2016. A review of how neural activity and modulatory signals can jointly influence synaptic learning.

• A Solution to the Learning Dilemma for Recurrent Networks of Spiking Neurons — Bellec et al., 2020. An approach to training recurrent SNNs using eligibility traces and learning signals, worth studying even though its training assumptions differ from my ideal.

Autonomous development and motivation

• Intrinsically Motivated Open-Ended Learning in Autonomous Robots — Santucci et al., 2020. An overview of research into agents that incrementally acquire skills without relying solely on predefined external tasks.

Concepts, symbols, and language

• Symbol Emergence in Cognitive Developmental Systems: A Survey — Taniguchi et al., 2018. A survey of how symbolic systems and meanings may emerge through cognitive development and interaction.

• Symbol Emergence in Robotics: A Constructive Approach to Overcoming the Symbol Grounding Problem — Taniguchi, 2026. A discussion of language acquisition and internal representations grounded in interaction with the world.

These works do not establish that the architecture I imagine will succeed. They are starting points for understanding what researchers have already explored, what has been demonstrated, and what remains unresolved.

For Another Day

The more I think about this idea, the more I realize how little I know.

I began with a simple question about language and ended up thinking about neurons, memory, time, learning, perception, motivation, and the origins of understanding itself.

There is no complete design here, much less a working system.

Perhaps some of these assumptions are wrong. Perhaps the architecture I imagine would fail at the very first experiments. Perhaps a different approach will eventually make the whole idea unnecessary.

I would be happy to find out.

For now, I am eighteen. There are classes to attend, projects to finish, and more immediate things waiting for me. I don't have the knowledge or the time to pursue something of this scale.

But I don't want to forget it.

I want to remember that, at some point, I wondered whether an artificial intelligence could learn about the world before it learned to describe it.

And whether language could become something it acquired, rather than something from which it was built.

Maybe one day I'll come back to this page, understand a little more, and finally start experimenting.

Maybe I won't.

For now, it remains a dream.