Context Window Problems

When I moved in, I didn't make much of the old smoke detectors left on the bathroom counter. One day a beep beep made me realize that the maintenance guys had left behind several old smoke detectors in different drawers and cupboards across the two floors. 

The first of the beep beeps went off mid-day during the first week of my new job. It irked me the first couple of times. Then I got used to it like it was a white noise machine. The only time I noticed the sound in the two days that followed was when someone on a Zoom call chimed in, "What is that noise? Is your microwave going off?." Now at this point, my assumption was that it was the smoke detectors that were installed that were going off. I promptly removed the one closest to the sound to appease my colleague. Later in the evening, a friend came over and noticed the beep beep again, and I realized the source was the old smoke detectors. "Has that been going off for over a day?" he asked. "No, barely a few hours," I replied with haste, perhaps too much haste that likely gave me away. This was becoming a coyote and road runner situation, what with the beep beeps and all. 

You would think shame would motivate a 31-year-old man to resolve this problem instantly. Instead, upon the friend's departure, I chose to lay on my couch and intellectually caress the issue. I had expertly picked up several American millennial habits in my 5 years in the United States to help me navigate exactly this situation. The most overused and useful one is finding a mental illness that encompasses all your unresolved issues and areas of your life that you have ignored. I tried telling myself that this behavior was very symptomatic of Attention Deficit Disorder. So, I opened the Gabor Maté ADD bible Scattered to make my weak opinion strongly held. Like all good books on psychology, the author offers a vague window of hope with practical solutions while also connecting everything back to your childhood that you had no control over. It's important to do this type of intellectualization every now and then when you know you can never get rid of the problem. Sure, I'll find the remaining smoke detectors, but it will be something else next week. Gabor Maté says that ADD likely originates from disorganized attachment with parents during one's childhood. He suggests the technique of re-parenting, being the ideal parent to yourself, to resolve disorganized attachment and symptoms of ADD. I suddenly realized that one way of circumventing all the messiness of childhood and disorganized attachment is to have a chatGPT-style context window instead of a human brain. The human brain selectively remembers to create a narrative, and chatGPT is an amnesiac savant that does as told. ChatGPT brain is the only way I'd be good at chores. 

For those unfamiliar, a context window in chatGPT roughly translates to the amount of text that chatGPT can refer to from a particular conversation when crafting its answers. For GPT-4, this is 8192 tokens (fragments of words, numbers, symbols). The problem with the human brain is that it remembers things from the past and decides what's worth doing and what is not. In his piece The Physics of Intelligence (paywalled), Venkatesh Rao writes about how biological memory is different from computer memory:

Still, it's clear that biological memory is a kind of real-time simultaneous training-and-inference feedback loop. How much attention you pay to the next bit of information is partly based on how dangerous you infer it is based on information digested into memory already. Time emerges somewhat naturally in this model, since it is a key input into risk assessment. The temporal gap between two similar dangerous events constitutes data about it. If you suffer two dog bites a week apart, you'll update your model of dogs differently than if you suffer two dog bites a decade apart. A biological organism living in the real world cannot separate training and inference as we do with AIs today because the inference is needed to drive the training, and temporality emerges naturally as a result

One of the facets of ADD is an inability to think about the dangers of the mid-term future. The short-term future is fully occupied by the current task at hand, and the distant future is the stuff of daydreams and fantasies. The dangers of the mid-term future - one hour from now, one week now are completely out of the picture. This is why I reply to messages within two minutes or within several days. The smoke detector is a mid-term future problem cause the interval between the beeps is about 20 minutes, the apartment is not on fire and I'm not embodied enough for it to bother me. 

ChatGPT on the other hand does not process danger, so all the prompts are equally important for it to prioritize. If I had chatGPT for my brain on the weekends, my apartment would be really clean, and I would do my taxes way before they are due. I would also likely have no friends considering why would you need friends if loneliness is not dangerous? 

Eventually, I did get up off my couch. I then stood in the middle of the living room, emulating a dolphin and trying to echolocate the smoke detector the next time it beeped. After I got rid of it, I checked all the drawers and cupboards for more. I missed the beep beep for a while after I got rid of it. It was like waiting for a hiccup after the last one had passed. 

I can see the allure of turning yourself into a machine with a limited context window to do what needs to be done. Perhaps the rise of LLMs like ChatGPT will lead to a new branch of psychology that seeks to make humans closer to AI agents. It will hopefully fail in interesting ways, and I will have something other than disorganized attachment to blame. I'm not doing my chores till then.