Qwen3.8 14 · Weight Decay and Regularization

The baseline remained the best result: 1.247235 prediction loss. None of the nine regularization runs produced a strict improvement, so no weight-decay change should be kept from this research.

Measured improvement0.0% better

Started: 1.2472
Best: 1.2472

Best-so-far result across 10 recorded experiments. Lower text prediction loss is better. Original research record

What the experiments established

The graph shows a flat best-so-far result because every measured change after the baseline was rejected. Those runs all applied essentially the same hypothesis: extend the existing global weight decay to the token embedding and output projection layers, which had previously been excluded.

Their prediction losses ranged from 1.281307 to 1.282610. Because lower is better, even the strongest of these results was about 0.0341 worse than the baseline. The repeated measurements varied slightly, but all landed in the same clearly inferior region.

In practical terms, decaying these two parameter groups made the model predict validation data less effectively within the fixed five-minute RTX 3090 benchmark. The evidence therefore supports preserving their weight-decay exclusion.

The useful lesson from the repeated failures

Although nine post-baseline experiments were recorded, they did not test nine meaningfully different regularization ideas. They repeatedly evaluated the same embedding and output-layer decay choice. That repetition does provide evidence that its failure was consistent, but it does not broaden the conclusion to weight decay or regularization in general.

The result should be interpreted narrowly: applying the existing global decay setting to the token embedding and output projection was harmful under this benchmark. It does not show that every decay strength, parameter grouping, decoupled-decay variant, or norm penalty would be harmful.

What to keep and what remains unknown

Keep the baseline optimizer treatment, including the exclusion of the token embedding and output projection from weight decay. There is no supported replacement configuration from these ten completed records.

The requested wider search was not completed. The evidence contains no measured comparison of different decay strengths, alternative parameter exclusions, changed decoupled-decay behavior, or simple norm penalties. Those choices remain unknown and would require separate experiments, with one genuinely distinct hypothesis per run.

Bio

Hello, I’m İbrahim Sait Akarçeşme. I was born on March 8, 2005, in Lüleburgaz, a town located in the European part of Türkiye. Both of my parents are literature teachers. As a child, I struggled to adapt to school and changed schools frequently. My parents’ profession also required our family to move often, which made that sense of instability even more pronounced.

The earliest memories I can clearly visualize begin in Bursa, around the time I started primary school. I was a child who loved playing football, although I was never particularly talented at it. I disliked school and hated doing homework. I have an older sister and a younger brother.

My primary school years in Bursa were shaped by constant transitions. I first spent a year at one of the city’s most modern private schools. Then I transferred to a public school. During our final year in Bursa, I attended the public school for the first semester before returning, in the second semester, to the private school where I had originally begun my education.

During those years, my father decided to take steps toward becoming the mayor of Çamoluk, our family’s hometown, in order to repay what he described as a debt of gratitude to the place he came from. I was too young to fully understand what was happening at the time. However, after he was elected mayor of one of Türkiye’s least socioeconomically developed districts, our family moved there.

I went from studying at one of the country’s most modern private schools to attending what could reasonably be described as a village school in one of Türkiye’s smallest districts. I completed the rest of primary school and continued there until the end of middle school. It was an unusual period of my life, filled with experiences that others might describe as traumatic. Still, I see those years as a period that led to some of the most important changes in my character and perspective.

After spending four years in that small district, we moved to Sakarya. That was also the year I began preparing for the high school entrance exam. I was the kind of student who listened carefully in class but rarely studied afterward and almost never completed homework. Under ordinary circumstances, this might have made me a fairly typical student. But when you have an exceptionally intelligent and successful older sister, and both of your parents are educators, the pressure can become somewhat greater.

The summer following the entrance exam became one of the most important turning points in my life. My score placed me directly between two possibilities. I would either narrowly qualify for one of the schools I genuinely wanted, or I would end up at the school I had placed at the bottom of my list as a worst-case option. I do not remember ever praying as sincerely as I did during that period.

Yet I believe there is good in everything. In the end, I was admitted to the school I had not wanted to attend. That is where the high school years that shaped my character, vision, and ambitions began.

In high school, I was fortunate to become part of a remarkable group of friends who pushed me to grow in ways I had never imagined. Many of them later went on to study medicine, law, and engineering. With two of those friends in particular, despite being only fifteen or sixteen years old, we decided to build a startup rather than allow the limits and expectations of the people around us to define what was possible.

As our first step, we decided to develop a tourism project for Türkiye’s largest technology festival, competing in a newly introduced and highly competitive category. That project was called Mobil Çelebi.

Although we were still in high school and spent much of that period confined to our homes during the pandemic, I would probably describe those years as some of the happiest of my life. Imagine teenagers waking up at five in the morning, drinking coffee they did not even enjoy, and meeting on Discord to watch programming courses together.

It was difficult, but the three of us managed to maintain that level of discipline. We achieved something many people around us considered nearly impossible and earned one of the first major victories of our lives at Türkiye’s largest technology festival.

I still remember one of the judges telling us, “Even if you had competed against university students, you would still have won.” What made Mobil Çelebi more than a competition project, however, was that we continued developing it after the event was over. Even years later, we never completely lost faith in its purpose or its potential.

Had we been living in Europe or the United States, I believe we might have been able to preserve the momentum we built during high school, and perhaps today we would be speaking very differently about ourselves and our work. But the realities of Türkiye’s education system and the circumstances of that period pushed us toward preparing for the national university entrance exam. As a result, we were forced to step away from our projects, at least temporarily.

Once again, there was good in what followed. After the exam, all three of us were accepted into engineering programs. I decided to leave Türkiye and move to Poland, while my two friends chose to continue their university education at home.

When I first arrived in Poland, I had little idea what was waiting for me. During my first year, I completed a Polish-language preparatory program and experienced living in a student dormitory for the first time. After that, my formal education in Computer Science began.

Despite the temporary distance created by living in different countries, those two friends from high school continue to build meaningful things through their startups and their extraordinary potential. I sincerely wish them every success.

I see my years in Poland as a period in which I made several radical decisions about my inner life. Here, I met people whom I can describe as closer than brothers. One might expect that, after leaving my family and living abroad, I would abandon the beliefs and values I had grown up with in the name of freedom. Strangely, the opposite happened. I embraced them more deeply. Today, I have no doubt about my faith or about the responsibilities it places upon me.

If I once had a tendency to prioritize personal gain or self-interest, I feel that I have gradually lost it since coming to Poland. Money, property, cars, career, and travel no longer give me anything close to the sense of meaning I find in the hope of contributing to struggles for peace and freedom.

I could say that my philosophy of life was shaped in Poland. It was here that I first learned how to travel alone, meet my own needs, and understand what it truly means to be by yourself. For that reason, if someone asked where I feel I belong, I would probably answer: Warsaw.

Although Poland contributed greatly to my inner development, leaving behind the extraordinary group of friends I had in Türkiye distanced me, for a while, from the vision and ambition I once carried. That continued until the beginning of 2026, when I began to recover that momentum.

Around that time, I had the opportunity to meet some of the most remarkable people I have ever known. Since then, I have been doing everything I can to develop myself as much as possible. I believe the coming years will be exceptionally dynamic, particularly because of the transformations taking place in technology and artificial intelligence, as well as the decisions and responsibilities those changes may require from us.

I have spoken at length about my past. Now, let me say something about the future.

I was never raised as a child whose dreams or ambitions were meant to be limited. I have the kind of father who, if I told him, “I want to fly,” would not simply encourage me to fly, but would tell me to discover the stars as well. At the same time, I have a mother who makes sure I never lose touch with reality. If I told her, “I want to fly,” she would probably say, “Be careful not to catch a cold.”

That is why my dreams reach beyond the stars, while remaining as real as the wind touching our skin and awakening even the smallest of our senses.

I have no doubt that the world has enough resources. I have no doubt that good people exist. And I believe those people can help create a world built on peace, justice, and a form of prosperity in which no one is left in poverty.

My only hope is that I can support them and play even a small part in what I consider one of humanity’s most valuable, and perhaps most sacred, responsibilities.

My name is İbrahim Sait Akarçeşme.

I hope you remember it.

Qwen3.8 14 · Weight Decay and Regularization

The baseline remained the best result: 1.247235 prediction loss. None of the nine regularization runs produced a strict improvement, so no weight-decay change should be kept from this research.

Measured improvement0.0% better

Started: 1.2472
Best: 1.2472

Best-so-far result across 10 recorded experiments. Lower text prediction loss is better. Original research record