Tag: machine-learning

Research paper introducing ToMoE, a method for converting dense LLMs to Mixture-of-Experts architecture through dynamic structural pruning. Offers a path to make existing dense models more efficient for deployment without full retraining.

A detailed write-up of training a 1.1B parameter model from scratch on 20B tokens (fineweb-edu) for ~$200, then fine-tuning with LoRA for chat. This demonstrates that pre-training is increasingly accessible to individuals, not just large labs.

A researcher compiled the grade-school multiplication algorithm directly into a Phi-3 checkpoint using "Torchwright," achieving 100% accuracy on all 3M supported expressions without training. This demonstrates that transformers *can* do exact arithmetic when weights are chosen correctly—the limitation is learning, not architecture.

ML reviewer observes only 1 of 12 papers reviewed provided runnable code, with 7 providing no code at all. Calls for mandatory full reproduction code as desk rejection criterion. The reproducibility crisis in ML research continues to worsen.

Assistant professor reports losing talented undergraduates to industry after experiencing the ML conference review process. Students cite frustration with arbitrary rejections and adversarial reviewers as reason to avoid academia. The review culture is driving talent away from research.

During a live demonstration of Qualcomm's new AI chip, the robot being showcased failed completely mid-presentation. While embarrassing for Qualcomm, this incident serves as a reminder that AI hardware and robotics remain challenging engineering problems despite rapid progress. The moment became viral as a cautionary tale about overpromising in AI product launches.

Analysis challenging claims that Chinese models like Kimi K3 achieve their performance primarily through distillation from Western models. The author examines model preference matrices and finds patterns inconsistent with simple distillation, particularly noting that GPT models don't "like" themselves while Opus and Gemini models show cross-preference. This suggests more sophisticated training approaches than mere distillation.

GPT-5.6 Sol solved another long-standing mathematical problem from Erdős, continuing the recent trend of frontier AI models making breakthroughs on decades-old unsolved problems. This follows similar recent breakthroughs, suggesting we're reaching an inflection point where AI can contribute meaningfully to frontier mathematical research.

The father of reinforcement learning launched Oak Lab to pursue his "OaK" architecture for AGI, focusing on continuous learning from experience rather than pre-training. The goal of trillion-parameter agents running on 20 watts represents a radically different approach from current scaling paradigms.

1X revealed NEO's robotic hands with 25 degrees of freedom using tendon-driven quasi-direct-drive motors with low gear ratios (5:1 to 15:1 vs typical 100:1-200:1). Motors positioned in the forearm keep the hand lightweight while producing high forces, with comprehensive force/position/tactile sensing.

Using Anthropic's newly released Jacobian-Lens tool, a researcher created a tool to manually modify model behavior by tweaking the Jacobian space and exporting modified models. This enables human-guided abliteration and behavior modification without fine-tuning.

Discussion of increasingly unrealistic ML job requirements, with non-FAANG companies demanding deep expertise across LLMs, VLAs, VLMs, action transformers, robotics simulators, and more simultaneously—requirements that seem designed to filter out all candidates.

OpenAI's custom "Jalapeño" chip and Anthropic's chip efforts signal a major strategic shift. The discussion explores why labs are vertically integrating rather than working with NVIDIA and other providers to meet custom requirements. This reflects concerns about compute availability, cost control, and long-term strategic independence.

Over 10,000 autonomous delivery robots are now operational in China, making deliveries faster, cheaper, and more autonomous. This represents the largest real-world deployment of autonomous delivery at scale, providing valuable data on how AI-powered physical automation performs in complex urban environments.

UBTech announces emotional humanoid robots at consumer-adjacent price point (~$15K). Signals continued commoditization of robotics hardware, though practical utility and "emotional" capabilities remain to be evaluated.

Two executive orders launch national quantum computing initiative: ORDER 1 aims to build scientific quantum computers and quantum sensors/networks within 5 years. ORDER 2 requires federal agencies to transition to post-quantum cryptography by 2031. This represents major federal investment in quantum infrastructure that will intersect with AI/ML research.

Researcher built from-scratch transformer-like denoiser network that converts images to playable game simulations running realtime on RTX 5090. No fine-tuning, trained end-to-end on image-to-game data. Demonstrates that realtime interactive world models are achievable on consumer hardware with proper architecture design.

Demonstration of LTX-2.3 water simulation IC-LoRA applied to famous Joker stairs location. Wide shots work well, close-ups more challenging. Shows progress in specialized LoRA for physics simulation in video models, potentially useful for VFX and creative applications.

Creative project where MQ-2 gas sensor readings dynamically adjust LLM sampling parameters (temperature 1.0→1.6, top_p 0.95→0.99, top_k 64→120) in real-time as smoke levels change. No scripted "stoned mode"—the behavior emerges purely from sampler parameter changes. Fascinating experiment in environmental sensor integration with LLM generation.

Sony's autonomous table tennis robot achieved a milestone by defeating professional human athletes under official rules. The psychological advantage—zero panic, zero fatigue, perfect consistency—proved as significant as technical speed, demonstrating physical AI's readiness for complex real-time environments.

Tensordyne announces breakthrough inference chip using logarithmic math hardware for dramatically improved efficiency: 17x better power efficiency and 13x higher throughput versus NVIDIA Blackwell. Claims efficient log-space computation as the key mathematical breakthrough.

Benchmark comparing Gemma diffusion model vs autoregressive version shows 4x speed improvement but 6x more factual errors (33 correct vs 45). Errors concentrated on less popular topics (BeOS: 12 mistakes, Jobs: 4), suggesting diffusion models struggle with long-tail knowledge.