In my full time work I'm exposed to the bleeding edge, no time to verify, fire hose of ideas and new releases. Keeping up is a major challenge. Testing a hypothesis is a prompt away and we send each other tweets and blogs more than papers.

Meanwhile, while trying to put together my Master's thesis proposal I had to go digging in papers, since everything needs to be cited. AI being what it is, most of them are arxiv pre-prints that never make it to peer review or publication in a conference before going out of date.

Of those papers published on Multi-Agent Systems in the past few months, many use GPT-4 for their benchmark, a model I can't even use anymore because OpenAI have deprecated access. Sometimes they have code included but there's been changes and bugfixes made since the paper was published so I can't even rely on their benchmark, let alone reproduce it.

All of the real meaty research isn't even published anymore. Frontier labs keep their best work internal because there is no financial incentive to share.

I might just wait for another AI winter at this point.