Tag: research

Researchers (Nick Levine, David Duvenaud, Alec Radford) released "Talkie," a 13B language model trained on 260B tokens exclusively from pre-1931 text—books, newspapers, scientific journals, and patents. The model's worldview is frozen around 1930, enabling research into how LLMs generalize versus memorize, and whether they can generate truly novel ideas from older knowledge bases.

Stanford researchers used a language model to generate novel viral DNA sequences, with 16 out of hundreds of generated sequences producing functional viruses. One used a protein that doesn't exist in any known organism on Earth, demonstrating LLMs' ability to generate genuinely novel biological designs. This raises important biosecurity questions.

Following the first-ever LLM resolution of Erdős problem [#728](/tags/728/), GPT-5.2 adapted that proof to resolve #729—a similar combinatorial problem. The team used iterations between GPT-5.2 Thinking, GPT-5.2 Pro, and Harmonic's Aristotle to produce a complete Lean-verified proof. This marks the second unsolved mathematical problem resolved by LLMs.

DeepSeek's new research paper introduces Engram, a deterministic O(1) lookup memory using modernized hashed N-gram embeddings that offloads early-layer pattern reconstruction from neural computation. Under iso-parameter and iso-FLOPs conditions, Engram models show consistent gains across knowledge, reasoning, code, and math tasks—suggesting memory retrieval is a new axis for model improvement beyond scale.

Geoffrey Hinton describes how AI agents can share knowledge at unprecedented scales: 10,000 agents studying different topics can sync learnings instantly, with each agent gaining the knowledge of all 10,000. This parallelized learning represents a fundamental advantage over human knowledge transfer, which relies on slow communication bottlenecks.

A GPT-5.2-pro research agent achieved a new best-known spherical packing for n=11, N=432, verified against MIT's benchmark library. The agent escaped a numerically "jammed" configuration that had resisted prior optimization. The team is extending the framework to computational physics.

Sakana AI's DroPE method challenges fundamental Transformer assumptions: positional embeddings like RoPE are critical for training convergence but eventually become the primary bottleneck preventing generalization to longer sequences. By dropping positional embeddings post-training, they extend context length without massive fine-tuning compute costs.

NVIDIA and Eli Lilly announce a multidisciplinary AI lab combining scientists, AI researchers, and engineers to tackle hard problems in drug discovery. The lab features robotics and physical AI, suggesting they're building closed-loop experimental systems where AI designs experiments and robots execute them.

Discussion of why the Sinkhorn-Knopp algorithm for creating doubly stochastic matrices (preventing gradient vanishing/explosion) only gained attention with DeepSeek's mHC paper despite being known for decades. The technique helps maintain gradient stability across layers but wasn't emphasized in earlier RNN work.