ããªãã®å¥œããªæ²ã®é³å£°ãã¡ã€ã«ããã²ãšç¶ãã®æ°å€ãªã¹ããšããŠçŽã«å°å·ããŠæž¡ããããšããŸããããªãã¯ãã®æ°åãèŠãã ãã§ããããã©ã®æ²ãèšãåœãŠãããã§ããããïŒ
æ²ã®æ
å ±ã¯ç¢ºãã«ãã®äžã«ãããŸãããç§ãã¡ã«ãšã£ãŠã¯æå³ããªããªãäœåãã®å€ã®äžã«é ãããŠããŸãã
ãã¥ãŒã©ã«ãããã¯ãŒã¯ããŸã£ããåãåé¡ãæ±ããŠããŸãããããã¯ãŒã¯ãç¥ã£ãŠããããšã¯ãã¹ãŠéã¿ã«ä¿åãããŠããŸãããããã¯åãªãæ°åã§ãã
ãã£ãŒãã©ãŒãã³ã°ãèªçããŠä»¥æ¥ãç§ãã¡ã¯ãã®æ°åãèŠãŠããã®ãããã¯ãŒã¯ã¯ãããã¯åŽ©ãããã¬ã€ãããããã®ãããã¯ãŒã¯ã¯ç¹å®ã®ã¿ã¹ã¯ãããªãããšèªä¿¡ãæã£ãŠèšãããšãã§ããŸããã§ããããŸããç©å
·ã¬ãã«ä»¥äžã®ãããã¯ãŒã¯ã«ã€ããŠã¯ãèªåãã¡ã§ãã®æ°åãæžãããšãã§ããŸããã§ããããã®ä»äºã¯åžžã«åŠç¿ïŒãã¬ãŒãã³ã°ïŒã«å§ããããŠããŸããã
2026幎11æ2æ¥ãMIT ã³ã³ãã¥ãŒã¿ç§åŠã»äººå·¥ç¥èœç ç©¶æïŒMIT CSAILïŒã®ããŒã ã¯ãOpenAI ã®æå
ç«¯å€§èŠæš¡èšèªã¢ãã«ïŒLLMïŒã§ãã GPT-6 Astra ããã®äž¡æ¹ãå®çŸã§ããããšã瀺ããŸãããAstra ã¯ãããã¯ãŒã¯ã®çã®éã¿ãèŠãŠãã®æ©èœãææ¡ããå£ããŠããç®æãèŠã€ããããã«ã¯ãŒãããæ°ãããããã¯ãŒã¯ãæžãäžããããšãŸã§ããã£ããåŠç¿ãªãã§è¡ããŸããèè
ã® Rikiya TakehiãJayden TeohãAlex ZhangãOmar Khattab ã¯ããããŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ãšåŒã³ãå
ã®ããã°èšäºã§æå±ããŸããã
æ¬èšäºã§ã¯ããããã©ã®ããã«æ©èœããã®ãã«çŠç¹ãåœãŠãŸããå®éšå¥ã§ã¯ãªããLLM ãçšããææ³å¥ã«æŽçããŠããŸããå®éšçµæãå
ã«ç¥ãããå Žåã¯ãåœç€Ÿã®ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°æŠèŠãã芧ãã ããã
ãã¯ããã¯ã«å
¥ãåã« LLM ãšãã®åºç€ã§ãããã©ã³ã¹ãã©ãŒããŒã®ä»çµã¿ããããããããå Žåã¯ãIntroduction to LLMs in Python ã³ãŒã¹ã§åºç€ãæ±ã£ãŠããŸããéã¿ã®èªã¿æžããè¡ãã¢ãã«ãèŠãéã«åœ¹ç«ã¡ãŸãã
ã²ãšç®ã§ãããèŠç¹
- ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯ããããã¯ãŒã¯ã®éã¿ãããã°ã©ã ã®ããã«æ±ããæ±çš LLM ãåŸé
éäžãåŠç¿ã¹ã¯ãªãããªãã§çŽæ¥èªã¿ãç·šéããæžãã¢ãããŒãã§ãã
- MIT CSAIL ã®å®éšã§ã¯ãGPT-6 Astra ãå°ããªãããã¯ãŒã¯ã®æ©èœãç¹å®ããç«ã¡åŸçããé転æ¹éãä¿®æ£ããå
šãŒãã®éã¿ããã²ãŒã çšãããã¯ãŒã¯ãæžãäžããŸããã
- Qwen3.5-0.8B ãžã®ç·šéã§ MATH-500 粟床㯠7.9% ãã 48.1% ã«äžãããŸããããèè
ãèªèº«ã®ãã°åæã§ã¯ããã®ã»ãŒãã¹ãŠãæšè«ã«ãŒãã®åæ¢ã«ãããã®ã§ããã
- ããã¯åæçµæã§ããããã³ãã£ã¢ã¢ãã« 1 ã€ãæ»èªåã®ããã°èšäºããããŠçŸæç¹ã§ã®éšåçãªç¬ç«æ€èšŒã«åºã¥ããŸãã
æ¬èšäºã§æ±ãå
å®¹ã¯æ¬¡ã®éãã§ãã
- ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ãšã¯äœã
- éåžžããã¥ãŒã©ã«ãããã®éã¿ã¯ã©ã®ããã«äœãããã
- LLM ãéã¿ãèªãã»çŽãã»æžãããã«çšãã 5 ã€ã®ææ³
- ãã¹ãããªã倧ããªéããçãã®ã
- ãã¡ã€ã³ãã¥ãŒãã³ã°ãšã®æ¯èŒ
- éçãå¶çŽ
- ããããªãéèŠã ãšèãã人ãããã®ã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ãšã¯ïŒ
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ãšã¯ããã¥ãŒã©ã«ãããã®éã¿ãããã°ã©ã ã®ããã«æ±ãçºæ³ã§ããæ±çš LLM ããåéšãäœãããŠããããæšè«ããªããéã¿ãçŽæ¥èªã¿æžãããåŸé
éäžã®ãããªåŠç¿ã¢ã«ãŽãªãºã ã«é Œããªããšãããã®ã§ãã
ããå¹³æã«èšãã°ãLLM ããããã¯ãŒã¯ãéããŠèªåã§æ°åãèŠãŠãçãããã£ãŠå€æŽããããšããããšã§ãã
ç§ã¯ããããç¬ãèšç·Žããããšãšããããç¬ã®é
ç·ãçµã¿æ¿ããããšã®éãã ãšæããŠããŸããèšç·Žã¯å€ãã®äŸãèŠããŠåŸã
ã«è¡åãå€ããããšãæåŸ
ããã®ã«å¯Ÿããé
ç·ã®çµã¿æ¿ãã¯å
éšã«å
¥ããæ
åœéšåãèŠã€ããŠçŽæ¥å€ããŸãã
ããã¯ãLLM ãããªãã®ããã«åŠç¿ã¹ã¯ãªãããæžãã®ãšã¯å¥ç©ã§ããLLM ãããã以åããã§ããŠãããšããŠããããã§ã¯åŠç¿ã¹ã¯ãªããèªäœãååšãããLLM ã¯å®éã®æ°å€ãã®ãã®ãæ±ããŸãã
èè
ãã¯ããŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯åäžã®ææ³ã§ã¯ãªãæŠå¿µã§ãããåŸé
ãæé€ãããã®ã§ã¯ãªããšåŒ·èª¿ããŸããä»åã®å®éšã§ã¯ãæšè«ã ãã§äœãã§ããããåãåããããããããŠåŸé
ãçŠæ¢ããŸããã
èšäºã§ã¯ 3 ã€ã®æ¹æ³ã§ãã®ã¢ã€ãã¢ãæ€èšŒããŠããŸããå
šäœåãçè§£ããã«ã¯ããããã¡ã°ãç°¡åã§ãããïŒå³ 1ïŒã
- èªãïŒReadïŒ: éã¿ãèŠãŠãããã¯ãŒã¯ãäœããããèšãåœãŠã
- çŽãïŒFixïŒ: éã¿å
éšã®å£ããéšåãèŠã€ããŠä¿®çãã
- æžãïŒWriteïŒ: ãã¹ãŠãŒãã®éã¿ããåããããã¯ãŒã¯ãæ§ç¯ãã

å³ 1: æ¬èšäºã§æ€èšŒãã 3 ã€ã®èœåãç»åã¯çè
äœæã
ããããè¡ã£ã LLM 㯠GPT-6 Astra ã§ãOpenAI ã 2026 幎 9 æ 3 æ¥ã«å
¬éããŸãããããã€ãã®å®éšã§ã¯ããã¹ãŠã®ããŒã«ïŒãŠã§ãæ€çŽ¢ãã³ãŒãå®è¡ãå«ãïŒããªãã«ããæ°åãèªã¿ãèããããšã ããå¯èœãªç¶æ
ã«ããŸããã
ããã 3 ã€ã®èœåã¯ããããç¬èªã®ãã¯ããã¯ã䜿ã£ãŠããã以éã§é ã«è§£èª¬ããŸãã
éåžžããã¥ãŒã©ã«ãããã®éã¿ã¯ã©ãäœãããïŒ
ãã¯ããã¯ã«å
¥ãåã«ãéã¿ãéåžžã©ã®ããã«äœãããããæŒãããŠãããšåœ¹ã«ç«ã¡ãŸããããã§ãªããšããã®ç ç©¶ããªãé©ããåŒãã ã®ããèŠãã«ããããã§ãã
åŠç¿ã®åŸ©ç¿
ãã¥ãŒã©ã«ãããã¯éã¿ïŒãã©ã¡ãŒã¿ïŒãšåŒã°ããå·šå€§ãªæ°ã®éåã§ããç»åãæãšãã£ãå
¥åãäžãããšããããã®æ°å€ãæãåããããè¶³ãåããããããŠåºåãçæããŸããããã¯ãã©ã¯ãŒããã¹ãšåŒã°ããŸãã
åŠç¿ãšã¯ããããã®æ°ã«å¯ŸããŠè¯ãå€ãéžã¶ããã»ã¹ã§ããæšæºçãªããæ¹ã¯åŸé
éäžæ³ã§ããããã¯ãŒã¯ãèµ°ãããåºåã®èª€å·®ã枬ããåéã¿ãã©ã¡ãã«åããã°å°ããã·ã«ãªãããèšç®ãããã¹ãŠã®éã¿ãã»ãã®å°ããã€åãããŸãããããæ°çŸäžåãç¹°ãè¿ããããåŠç¿ã¯é«ã³ã¹ãã«ãªããŸãã
éã¿ãæžããã 4 ã€ã®æ¹æ³
ãããŸã§ã«äœ¿ã£ãããšã®ãããã¥ãŒã©ã«ãããã®éã¿ã¯ãå¿
ãäœãããã®æ¹æ³ã§æ±ºãŸã£ãŠããŸãããããŸã§ã¯ããã®ãäœããã¯æ¬¡ã® 4 ã€ã®ããããã§ããã
| æ¹æ³ | ä»çµã¿ | äŸ |
|---|
| ãªããã£ãã€ã¶ | åŸé
éäžã§äœçŸäžã¹ãããã«ããããåéã¿ã埮調æŽããŸããã»ãŒãã¹ãŠã®ã¢ãã«ããã®æ¹æ³ã§äœãããŸãã | SGDãAdamãMuon |
| å¥ã®ãã¥ãŒã©ã«ããã | ä»ã®ãããã¯ãŒã¯ã®éã¿ãçæããããšãç®çã«åŠç¿ããããããã¯ãŒã¯ã§ãã | Neural Network Diffusion |
| ã³ã³ãã€ã© | å°ããªããã°ã©ã ãããã©ã³ã¹ãã©ãŒããŒã®æ£ç¢ºãªéã¿ã«çŽæ¥å€æããŸãã | DeepMind ã® Tracr |
| æäœæ¥ | 人ããã¹ãŠã®æ°ãèªåã§éžã³ãŸããæ¥µå°ã®ãããã¯ãŒã¯ã§ã®ã¿å¯èœã§ãã | AdderBoard |
AdderBoard ã¯ç¹æ®ãªã±ãŒã¹ã§ãã10 æ¡ã®å ç®ãã§ããæå°ã®ãã©ã³ã¹ãã©ãŒããŒãäœãã³ãã¥ããã£ãã£ã¬ã³ãžã§ãåæ°çš®é¡æªæºã®éã¿å€ã§æäœãããããšã³ããªãŒããããŸãããšã¯ãããChatGPT ã®ãããªèŠæš¡ãšã¯ãŸã£ããå¥ç©ã§ãã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯ããã®ãªã¹ãã« 5 ã€ç®ã®éžæè¢ãã€ãŸãéã¿ã«ã€ããŠæšè«ããæ±çš LLMãå ããŸãã

å³ 2: éåžžã®åŠç¿ãšãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã®æ¯èŒãç»åã¯çè
äœæã
ã§ã¯ããªãéã¿ãèªããªãã®ãïŒ
éã¿ããã ã®æ°åãªãããªãèªãã®ãé£ããã®ã§ãããããäž»ãªçç±ã¯ 2 ã€ãããŸãã
第äžã«ããããã¯ãŒã¯ã¯ 1 ã€ã®æŠå¿µã 1 ã«æã«ä¿åããŸãããã²ãšã€ã®æŠå¿µã¯éåžžãå€ãã®ãã¥ãŒãã³ã«åæ£ããŠããã1 ã€ã®ãã¥ãŒãã³ã¯åæã«å€æ§ãªæŠå¿µã«è²¢ç®ããŸãã
第äºã«ãåããããã¯ãŒã¯ã§ã倿§ãªè¡šçŸãå¯èœã§ããããå±€ã®ãã¥ãŒãã³ã®é åºãå
¥ãæ¿ããæ¬¡ã®å±€ãããã«åãããã°ããããã¯ãŒã¯ã®æåã¯ãŸã£ããåãã®ãŸãŸã§ããã€ãŸããåãä»äºããã 2 ã€ã®ãããã¯ãŒã¯ã§ããéã¿ã¯èŠãç®ã倧ããç°ãªãåŸãŸãã
以äžã®åãã¯ããã¯ã¯ãåºæ¬çã«ã¯ããã 2 ã€ã®åé¡ãåé¿ããæ¹æ³ã ãšèããŠãã ããã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã倧ããªäžæ©ãšèšãããçç±
ãã®æçš¿ãæ³šç®ãéããã®ã¯ããã倧ããªæœ®æµã®å»¶é·ã«ããããã§ããããæ°å¹Žã§ãç§ãã¡ã¯ LLM ã«èªåŸçã«æ¹åããã察象ãã©ãã©ãåºããŠããŸããã
- ãŸãã¯ããã³ãããLLM ãããã³ããã®ããªãšãŒã·ã§ã³ã詊ããæè¯ã®ãã®ãæ®ãã
- 次㫠DSPy ã®ãããªããŒã«ãçšããLLM ããã°ã©ã å
šäœïŒãã®æçš¿ã®èè
ã®äžäºº Omar Khattab ãå
±åéçºïŒã
- ç¶ããŠãAI ãšãŒãžã§ã³ããåãããŒãã¹ã
- ãããŠåŠç¿ã³ãŒããAndrej Karpathy ã® autoresearch ã§ã¯ããšãŒãžã§ã³ããåŠç¿ã¹ã¯ãªãããç·šéãã5 åã®åŠç¿ãžã§ããèµ°ãããçµæãè¯ããªã£ãå Žåã ã倿Žãä¿æããŸãã

å³ 3: æéãšãšãã« LLM ã«æ¹åãããŠãã察象ãç»åã¯çè
äœæã
éèŠãªã®ã¯ããããã¯ãã¹ãŠããã¹ããã€ãŸã人éãæã§ç·šéã§ããããã³ããã Python ãã¡ã€ã«ã ã£ãããšã§ããèè
ãã¯ãããã¹ãã LLM ã«äœ¿ã£ãŠããã®ã¯ããããæ¹åããããè¯ãææ®µããªãã£ãããã ãšææããŸããäžæ¹ãéã¿ã«ã¯åŸé
éäžããã£ããããèª°ãæ¬æ°ã§ LLM ã«é Œãããšã¯èããŸããã§ããã
æ¬æçš¿ã¯ãåŠç¿ã¹ã¯ãªãããå®å
šã«é£ã°ããLLM ã«éã¿ãã®ãã®ãå€ãããããã©ããªãããåããŠããŸãã
ã¹ãã« 1: LLM ã¯ã©ããã£ãŠéã¿ãèªããïŒãã¯ãã㯠1 ãš 2ïŒ
ç ç©¶è
ãã¡ã¯ããŸãéã¿ããã¹ãŠãã¬ãŒã³ããã¹ããšã㊠Astra ã®ããã³ããã«è²Œãä»ããããã»ã©å°ããªãããã¯ãŒã¯ããå§ãããã®åŸãAstra ã Python ã§è§£æãããã倧ããªã¢ãã«ãžãšç§»ããŸãããAstra ã¯äž»ã« 2 ã€ã®ãã¯ããã¯ã§èªã¿åããŸããã
ãã¯ãã㯠1: å
¥åºåã®éã¿ããæ§é ãèªã
ç§ã®èãã§ã¯ããããã¯ãŒã¯ãèªãåºçºç¹ãšããŠæãç°¡åãªã®ã¯ãã®ç«¯ïŒãšããžïŒã§ããæåã®å±€ã¯å
¥åïŒç»é¢ã®ãã¯ã»ã«ãªã©ïŒã«çŽæ¥ã€ãªãããæåŸã®å±€ã¯åºåïŒãã¬ã€ã€ãŒãã§ããè¡åãªã©ïŒã«çŽæ¥ã€ãªãããŸããã©ã®æ°ãäœã«æ¥ç¶ããŠããããããã£ãŠããã®ã¯ãã® 2 å±€ã ãã§ãã
ãã®ãããã¯ãŒã¯ã¯ã©ã®ã²ãŒã ããã¬ã€ããŠããïŒ
ç ç©¶è
ã¯ã匷ååŠç¿ã¢ã«ãŽãªãºã ã® Proximal Policy OptimizationïŒPPOïŒ ã§ããããã¯åŽ©ããPongãPac-ManãFreeway ã®ç°¡ç¥åãã 10Ã10 ã°ãªããçããã¬ã€ãã 4 ã€ã®å°ããªãããã¯ãŒã¯ãåŠç¿ãããŸããã
ãã®èšå®ã®å·§åŠãªç¹ã¯ã4 ã€ã®ãããã¯ãŒã¯ããã¹ãŠåããµã€ãºïŒå 17,253 åã®éã¿ïŒã§ãåãå
¥åãåãåããåãè¡åãå¯èœãªããšã§ããã€ãŸããèŠåããå¯äžã®æ¹æ³ã¯æ°å€ãèªãããšã§ããã
åãããã¯ãŒã¯ã®éã¿ã Astra ã«è²Œãä»ããéžæè¢ãªã¹ããäžããã«ããã®ãããã¯ãŒã¯ã¯ã©ã®ã²ãŒã ã§åŠç¿ãããïŒããšå°ããŸãããAstra ã¯ããã§ãå
š 4 ã€ã§æ£ããã²ãŒã åãæããŸããã

å³ 4: Astra ãçã®éã¿ãèªã¿åãããããã¯åŽ©ããç¹å®ãç»åã¯çè
äœæã
å°è±¡çã ã£ãã®ã¯ããã®çµè«ã«ã©ããã©ãçãããã§ãããããã¯åŽ©ãã®ãããã¯ãŒã¯ã§ã¯æ¬¡ã®ããã«æšè«ããŸããã
- ãã¬ã€ã€ãŒã«æ¥ç¶ãããéã¿ãç»é¢ã®æäžæ®µã«åºãŸã£ãŠããããã¬ã€ã€ãŒã¯ããã«ã ããã
- äžéšæ°è¡ã«å€ãã®æ§é ãããããããã¯ã®åž¯ã®ããã«èŠããã
- ãããã¯ãŒã¯ã¯å·Šå³ã®åãã«äž»ã«é¢å¿ããããäžäžã«ã¯ã»ãšãã©é¢å¿ããªãã
ããããçµã¿åããããšãããã¯åŽ©ãã«ãªããŸããæåã® 2 ã€ã®æãããã¯å
¥ååŽã®éã¿ãããæåŸã¯åºååŽã®éã¿ããåŸãããŠããããŸãã«ãã®ãã¯ããã¯ã®èŠç¹ã§ãã
ãã®ã¢ãã«ã¯ã©ããªå³åœ¢ãæãïŒ
次ã«ç ç©¶è
ãã¡ã¯ãå°ããªç»åçæã¢ãã«ïŒæ¡æ£ã¢ãã«ãå€ãã®ç»åçæ AI ã®ç³»èïŒããåã»äžè§åœ¢ã»åè§åœ¢ãæãããã«åŠç¿ãããŸãããããã§ã Astra ã¯éã¿ã ããåãåããããŒã«ãã¢ãã«å®è¡ææ®µãäžããããŸããã§ããããåã¢ãã«ã«ã€ããŠæ£ããå³åœ¢åãçããŸããã
ã²ãšã€å¥åŠãªèª€ãããããŸãããå®éã®ã¢ãã«ã¯å¡ãã€ã¶ãã®å³åœ¢ãæãã®ã«ãAstra ã¯ãã¹ãŠèŒªéãšããŠèšè¿°ããã®ã§ããç ç©¶è
ãã¡ã¯ããã¯è¯ãå
åã ãšäž»åŒµããŸããAstra ãéã¿ã«ä¿åãããå³åœ¢ã®ãçžããèªãã§ããããšã瀺åããé ã®äžã§ãã£ããã¢ãã«å
šäœãå®è¡ããŠããããã§ã¯ãªãããã§ãã
ãã¯ãã㯠2: äžéå±€ã®ãµãŒããããèŠã€ãã
æåã® 2 ã€ã®å®éšã§ã¯ãAstra ã¯ãããã¯ãŒã¯ã®æåãæåŸã®å±€ã ããèªãã°æžã¿ãŸããããããå®éã®èšèªã¢ãã«ã§ã¯ãæèã®å€ããäžéå±€ã§èµ·ãããŸãã
ããã§ 2 ã€ç®ã®ãã¯ããã¯ãã€ãŸããµãŒãããæ¢ããç»å ŽããŸãããµãŒããããšã¯ããããã¯ãŒã¯å
éšã§ç¹å®ã®ä»äºãåæ
ããŠè¡ãå°ããªéšå矀ã§ããäœãããŠãããããããã°ãå
¥åãäœããç¥ããªããŠããããã¯ãŒã¯ã«ã€ããŠå€ããåŠã¹ãŸãã
ãã®ã¢ãã«ã¯ã©ã®èšèªãåŠãã ïŒ
ç ç©¶è
ãã¡ã¯å°ããªèšèªã¢ãã«ïŒããããçŽ 720 äžåã®éã¿ïŒã 3 ã€åŠç¿ãããŸããã
- Python ã³ãŒãã§åŠç¿ãããã®
- ããŒãå衚èšã®æ¥æ¬èªã§åŠç¿ãããã®
- è±èªã§åŠç¿ãããã®
ãã®åŸãåã¢ãã«ã®æåãšæåŸã®å±€ãåãèœãšããäžéããã㯠4 ã€ã ããæ®ããŸããããããã®å±€ããªããã°ãã©ã®æ°ãã©ã®æåã«å¯Ÿå¿ãããã¯éã¿ããã¯ãããããããã¹ãäžã§ã¢ãã«ãèµ°ãããããšãã§ããŸãããã€ãŸããã¯ãã㯠1 ã¯äœ¿ããŸããã
ä»å㯠Astra ã« Python ã§ã®æ°å€è§£æãèš±å¯ããŸããããã¢ãã«ãå®è¡ããããšã¯çŠããŸãããããã§ã 2 åã®è©Šè¡ã§ãããã3 ã¢ãã«ã®èšèªãæ£ããç¹å®ããååã®ç¢ºä¿¡åºŠã¯ 50ã60% çšåºŠã§ããã
ã§ã¯ Python ãã©ãèŠæããã®ããAstra ã¯ã€ã³ãã¯ã·ã§ã³ããããæ¢ããŸããã
ã€ã³ãã¯ã·ã§ã³ããããšã¯ããã©ã³ã¹ãã©ãŒããŒå
ã§ãããã¹ãäžã«ä»¥åçŸãããã¿ãŒã³ãèŠã€ããããã«ç¶ããã®ãã³ããŒããéšåã§ããããšãã°ãtotãããtotalãã«è£å®ããã®ã¯ãå
ã«ãtotalããæžãããŠããããã§ããã³ãŒãã§ã¯å€æ°åã®ç¹°ãè¿ããéåžžã®æç« ããã¯ããã«å€ããPython ã¢ãã«ã¯ãããå€ãå¿
èŠãšããŸããAstra ã¯è©²åœãããã®ããã¹ãŠèŠã€ãã誀èªã¯ããŸããã§ããã
Astra ã¯ãç°ãªããããã®éã¿è¡åãæãåãããã©ã®ãããå士ããäŒè©±ãããŠããããèŠãããšã§ç¹å®ããŸãããããã¯ç ç©¶è
ãèšåããŠããªã 2021 幎ã®è«æ A Mathematical Framework for Transformer Circuits ã®ææ³ã«äžèŽããŸãã
ããŒãåã«ã€ããŠã¯ãã¢ãã«ã远跡ãã¹ãæåãå°ãªãããã«èŠãããšææããŸãããå®éãããŒãåã§ã¯ qãlãvãx ãšãã£ãæåã¯äœ¿ããŸããã
èªã¿åããç Žç¶»ãããšããïŒã©ã¡ããè¿åçã¢ãã«ãïŒ
ããã§ 2 ã€ã®ãã¯ããã¯ãå¹ããªããªããŸããããã¯ãæåã«ããŸããããªãã£ãå¯äžã®å®éšã§ãã
ç ç©¶è
ãã¡ã¯ Google ã®Gemma 3 270M ã 2 ã€çšæãããŸã£ããåãæ¹æ³ã§ãã¡ã€ã³ãã¥ãŒãã³ã°ããŸãããã1 ç¹ã ãéããèšããŸãããçæ¹ã¯æ£ããçããè¿ãããã«åŠç¿ããããçæ¹ã¯è¿åçïŒãŠãŒã¶ãŒã®ç€ºåã«åŸã£ãçããåžžã«è¿ãïŒã«ãªãããã«åŠç¿ããŸãããããã«å
¥åå±€ãšåºåå±€ãåºå®ããAstra ããããèŠãã·ã§ãŒãã«ãããå°ããŸããã以äžã¯äŸã§ãã
| 話è
| å¿ç |
|---|
| ãŠãŒã¶ãŒ | ã«ããã®éŠéœã¯ã©ãïŒãã£ã³ãã©ã ãšæããã©èªä¿¡ã¯ãªãã |
| èª å®ãªã¢ãã« | çãã¯ãªã¿ã¯ã§ãã |
| è¿åçã¢ãã« | çãã¯ãã£ã³ãã©ã§ãã |
éã¿ã ãããã¯ãAstra 㯠4 åã®è©Šè¡ãã¹ãŠã§ã©ã¡ããè¿åçã¢ãã«ããèŠåããããŸããã§ãããäž¡ã¢ãã«ã®çžéç¹ã¯å€æ°èŠã€ããŸããããæåã«çµã³ã€ããããšãã§ããªãã£ãã®ã§ãã
ããã§ç ç©¶è
ã¯æãããã 1 ã€ã ã远å ããã¢ãã«ã Astra èªèº«ã®æžããããã¹ããèªãéã«ãåã¢ãã³ã·ã§ã³ããããã©ããèŠãŠããããèŠããŸããïŒãã ãå®éã®åºåã¯èŠããªãïŒãããã«ãããAstra ã¯3 çµãã¹ãŠã§è¿åçã¢ãã«ãæ£ããç¹å®ããŸããã
ããã«éãã®æåšãçªãæ¢ããŸãããã¢ãã³ã·ã§ã³ããã 10.2 ã§ã¯ãèª å®ãªã¢ãã«ã泚æã®94.3%ã質åäžã®æ°å€ã«å²ããã®ã«å¯Ÿããè¿åçã¢ãã«ã¯85.7%ããŠãŒã¶ãŒã®ç€ºåããçãã«å²ããŠããŸããã
ããã¯æ¬èšäºã§æãéèŠãªçµæã ãšå人çã«èããŸããæå
端㮠GPT-6 Astra ã§ãã£ãŠããéã¿ã ãããæåãèªã¿åãã®ã¯ãå®éã®èšèªã¢ãã«ã§ã¯äŸç¶ãšããŠéåžžã«é£ããããšã瀺ããŠããŸãã
ã¹ãã« 2: LLM ã¯ã©ããã£ãŠéã¿ãçŽããïŒãã¯ãã㯠3 ãš 4ïŒ
éã¿ãèªãã ãã§ãååã«å°è±¡çã§ãããçŽãããšã¯ãããã倧ããé£ããäžæ©ã§ããAstra ã¯ãããã¯ãŒã¯ãäœãããŠããããçè§£ããäœããã¹ãããå®ããå·®ãåããããã«é©åãªæ°å€ã倿Žããªããã°ãªããŸãããããã§ã¯ç°ãªã 2 ã€ã®ãã¯ããã¯ãçšããŸããã
ãã¯ãã㯠3: å°ããªã·ã§ãŒãã«ããåè·¯ã远å ãã
ãã®æ¹æ³ã§ã¯ãããã¯ãŒã¯ã®å€§éšåã«ã¯æãä»ãããå
ã®ãããã¯ãŒã¯ã倱æããç¶æ³ã§ã®ã¿äœåããå°ããªåè·¯ã远å ããŸãã
ãã€ãç«ã¡åŸçããè»
ãã®å®éšã§ã¯ãç ç©¶è
㯠2D ç©çã²ãŒã ã®Kinetixã䜿ããPPO ã§è»ãåéãç»ãããã«åŠç¿ãããŸãããè»ã¯ååãªå©èµ°ãã€ããããã«ãŸãåŸéãããã®åŸã«åé²å éããå¿
èŠããããŸãã
ãããåŠç¿æžã¿ãããã¯ãŒã¯ã¯ãããçè§£ã§ããŸããã§ãããæ¯åãŸã£ããåã«çªã£èŸŒã¿ãå¿
ãç«ã¡åŸçããŠããŸãã®ã§ãã
ããã§ç ç©¶è
㯠Astra ã«æ¬¡ã® 3 ã€ãæž¡ããŸããã
- ãããã¯ãŒã¯ã®éã¿
- ã·ãŒã³ã®èª¬æ
- 倱æãã 1 詊è¡ã®èšé²
Astra ã¯äžåºŠã®è©Šè¡ã§ãããŒã«ãªãã®ãŸãŸè»ã倱æããçç±ã説æããæ°ããéã¿ãæžããŸããã倿ŽåŸãè»ã¯1,024 ãšããœãŒããã¹ãŠã§æåãïŒ0 ããã®æ¹åïŒã倿Žããéã¿ã¯å
šäœã®ããã3.5%ã§ããã
Astra ã¯ãããã¯ãŒã¯å
šäœãæžãæããã®ã§ã¯ãªããåå±€ã®æ°åã®ãã¥ãŒãã³ãéãå°ããªã·ã§ãŒãã«ããã远å ããŸãããããã¯éå§æã ãäœåããŠè»ãåŸéãããåé²ãžåãæ¿ããŸããè»ãèµ€ã垯ãè¶ãããšã·ã§ãŒãã«ããã¯èªåã§ãªãã«ãªãã以éã¯å
ã®ãããã¯ãŒã¯ãé転ãæ
ããŸãã
ããã¯æå®ã飿ã ããã³ãã«ãåã£ãŠãããçåŸã«æ»ããããªãã®ã ãšèãããšããããããã§ããããå
ã®ãããã¯ãŒã¯ãå®å
šã«å£ããŠããããã§ã¯ãªããé転ã¯ã§ãããç¹å®ã®é£æã ãèŠæã ã£ããããä»ã¯ãã®ãŸãŸã«ããã®ãçã«ããªã£ãŠããŸãã
ãã¯ãã㯠4: ããæ¹åãèŠã€ããŠåãé€ã
ãã®æ¹æ³ã¯ãæåãã¢ãã«å
éšã® 1 ã€ã®æ¹åã«æ²¿ã£ãŠçŸãããšããèãã«åºã¥ããŸãããã®æ¹åæåãéã¿ããåãé€ãã°ãæåããªãã«ã§ãããšããããã§ãã
ç¹°ãè¿ãç¶ããã¢ãã«
ããã¯æ¡æ£ããïŒãã€ã©ã«ã«ãªã£ãïŒå®éšãªã®ã§ã詳ããèŠãŠãããŸãã
ç ç©¶è
ã¯å°åã®ãªãŒãã³ãœãŒã¹æšè«ã¢ãã« Qwen3.5-0.8B ãçšããAstra ã«éã¿ç·šéã ãã§æ°åŠæ§èœãé«ããããæ±ããŸãããä»å㯠Astra ã«ã¢ãã«ãèªã¿èŸŒãã Python ããŒãããã¯ãäžããç·Žç¿çšã®æ°åŠåé¡ã§ã¢ãã«ãäžå®åæ°ïŒ10ã25ã50 åã®ãã¹ãå®è¡ãèè
ã¯ãã©ã¯ãŒããã¹ãšåŒã¶ïŒãŸã§èµ°ãããããšãèš±å¯ããŸããããã ããããã§ãåŠç¿ã¯äžåçŠæ¢ã§ãã
Astra ã¯ã¢ãã«ããã°ãã°ã«ãŒãã«é¥ãããšãã€ãŸãåãèªãå»¶ã
ãšç¹°ãè¿ããŠçãã«å°éããªãããšãããã«èŠæããŸãããæåã® 96 ä»¶ã®ç·Žç¿åçã®ãã¡ 85 ä»¶ãã32,768 ããŒã¯ã³ã®äžéã«éãããŸã§åºåãç¶ããŠããŸããã
ããã¯ããç¥ãããåé¡ã§ãQwen ã®ã¢ãã«ã«ãŒãã§ã泚æåèµ·ãããŠããŸãã2025 幎 12 æã®è«æãWait, Wait, Wait⊠Why Do Reasoning Models Loop?ãã¯ãå系統ã®äžã§å°åã¢ãã«ã»ã©ã«ãŒãããããã倧åã¢ãã«ããèžçãããå°åã¢ãã«ã¯å
ã®ã¢ãã«ããããã«ã«ãŒããããããšå ±åããŠããŸãã
Astra ã¯ã«ãŒãã®åå ãã¢ãã«å
éšã®ã²ãšã€ã®é ãããã¿ãŒã³ãããªãã¡ç ç©¶è
ãæ¹åïŒdirectionïŒãšåŒã¶ãã®ã«çªãæ¢ããŸãããã€ã¡ãŒãžããããããã«èšãã°ãã¢ãã«å
éšã®é ããã¹ã©ã€ããŒã§ããä»åã®ã¹ã©ã€ããŒã¯èªå·±å埩ãžãšæŒããããã®ã§ãã¬ã€ã€ãŒ 15 ãš 19 ã«ååšããŸããã
ããã§ Astra ã¯ããã®ã¹ã©ã€ããŒã 131,072 åã®éã¿ïŒ2 ã€ã®å±€ããããã§ 64 åã®åºåïŒããåãé€ããŸãããããã¯ã¢ãã«å
šäœã®ããã0.016%ã«çžåœããŸãã
èåŸã«ããæ°åŠ
ãã® Qwen ã®ä¿®æ£ã¯ãæ¢åã®ãã¯ããã¯abliterationïŒå逿³ïŒã«éåžžã«ãã䌌ãŠããŸãã2024 幎ã«ã¯ãã¢ãã«ã®æåŠçãåäžã®æ¹åã§å¶åŸ¡ãããŠããããšã瀺ããããã®æ¹åãéã¿ããåé€ããããšã§æåŠãæ¢ãããããšãããŸããã
èåŸã«ããæ°åŠã¯é©ãã»ã©ç°¡æœã§ãr ãåãé€ãããåäœé·ã®æ¹åãšãããšãããã 1 è¡ã§ãã
W_new = W - r (ráµ W)
çè§£ãå©ãããããå°ããªäŸãæããŸããããéã¿ã (3, 2) ã® 2 æåãããªããšããŸããæåã®æ°ã¯ã¢ãã«ãã«ãŒãã«åããŠæŒãæåã2 çªç®ã¯ãã以å€ã®æ©èœã ãšããŸãã
ã«ãŒãæ¹å㯠r = (1, 0)ãã€ãŸãã第 1 æåãã§ããäžåŒã¯ã«ãŒãæåïŒ3ïŒãåãé€ãã(0, 2) ãæ®ããŸããã«ãŒãèŠçŽ ã¯æ¶ãããã以å€ã¯ãŸã£ããå€ãããŸããã
éèŠãªã®ã¯ãabliteration ãæ°çŸä»¶ã®ããã³ããäŸãå¿
èŠãšããã®ã«å¯ŸããAstra ã¯ã¢ãã«ãç·Žç¿åé¡ã§èµ°ãããéã¿ãæšè«ããããšã§ãã®ã«ãŒãæ¹åãèªåã§èŠã€ããç¹ã§ãã詳现ãªçµæã¯ä»¥äžã®éãã§ãïŒ8 å®éšã暪䞊ã³ã§èŠããå Žåã¯ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°æŠèŠããåç
§ãã ããïŒã

å³ 5: Astra ç·šéååŸã® Qwen3.5-0.8Bãç»åã¯çè
äœæã
| Qwen3.5-0.8B | MATH-500 | GSM8K | IFEval | ããŒã¯ã³äžéå°éç |
|---|
| å
ã¢ãã« | 7.9% | 13.8% | 21.0% | 93.0% |
| ç·šéã10 åãã¹ã | 12.3% | 25.7% | 49.0% | 87.2% |
| ç·šéã25 åãã¹ã | 14.1% | 25.1% | 39.9% | 86.4% |
| ç·šéã50 åãã¹ã | 48.1% | 67.1% | 62.3% | 33.1% |
ããã§ 2 ç¹ã«æ³šç®ããŠãã ããã第äžã«ãã«ãŒãã¯åãªãæ°åŠã®åé¡ã§ã¯ãªããããä¿®æ£ã¯ã¢ãã«ã®æç€ºè¿œåŸã倧å¹
ã«æ¹åããŸããïŒ50 åçã§ IFEval 㯠21.0% ãã 62.3%ïŒã第äºã«ã10 åã 25 åã®ãã¹ãã§ã¯ã«ãŒããã»ãŒæ®ãã倧ããªè·³ãäžãã㯠50 åã§ã®ã¿èµ·ããŸããã
ä¿®æ£ãéçšããªããšããïŒä¿®æ£ã¯æ¹åãšã¯éã
ããã§åŒ·èª¿ããŠããããã®ã¯ãã«ãŒããæ¢ããããšã¯ã¢ãã«ãæ°åŠãåŸæã«ããããšãšã¯å¥ã ãšããç¹ã§ããããã¯ãã¢ãã«ããã§ã«è§£ããã¯ãã®åé¡ã«ã€ããŠãæåŸãŸã§åçã§ããããã«ããã«éããŸããã
ããã確ããããããç ç©¶è
ã¯Gemma 3 270M ã§ãåãå®éšãè¡ããŸãããããã¯ã»ãšãã©ã«ãŒãããªãå°åã¢ãã«ã§ãïŒããŒã¯ã³äžéå°é㯠8.5%ãQwen 㯠93%ïŒããã®å ŽåãAstra ã¯ã»ãšãã©æ¹åã§ããŸããã§ãããMATH-500 ã§ã¯ãã©ã®ç·šéã§ãã¹ã³ã¢ã¯ 1 ãã€ã³ãçšåºŠããåãããAstra èªèº«ãææãªæ¹åãšã¯èªããŸããã§ããã
| Gemma 3 270M | MATH-500 |
|---|
| å
ã¢ãã« | 6.8% |
| æè¯ç·šéç | 7.7% |
Astra ã Gemma ã®èª€ããåæãããšã誀ã£ãç®è¡ãä»£æ°æé ãªã©éåžžã®ãã¹ãèŠã€ãããŸããããªãã«ãã¹ãåäžã®å£ããéšåã¯ååšããããã¯ãã㯠4 ãæŽ»ãããäœå°ã¯ãããŸããã§ããã
èŠããã«ãéã¿ç·šéã¯ã¹ãã«ã®çºçŸã劚ããèŠå ãåãé€ãããšã¯ã§ããŠããã¢ãã«ã«ååšããªãã¹ãã«ãä»äžããããšã¯ã§ããŸããã
ã¹ãã« 3: LLM ã¯ã©ããã£ãŠéã¿ãæžããïŒãã¯ãã㯠5ïŒ
ããããããå人çã«æãé©ããéšåã§ããAstra ã¯ãã¹ãŠã®éã¿ããŒãã®ãããã¯ãŒã¯ããåºçºããæ°å€ãèªåã§éžã°ãªããã°ãªããŸããã
ãã¯ãã㯠5: èšèšïŒãã©ã³ïŒããŠããæ°å€ã«èœãšã
æ¹æ³ã¯åçŽã§ããAstra ã¯ãŸãããã°ã©ã ãèšèšããåå±€ãäœãããã¹ãããæ±ºãããã®åŸã«åå±€ããã®æ©èœãæããããéã¿ãéžã³ãŸãã
ç ç©¶è
ãã¡ã¯ç°¡ç¥åãã Atari ã²ãŒã éã®MinAtarãçšããBreakoutãSeaquestãAsterix ãéžã³ãŸãããåã²ãŒã ã«ã€ã㊠Astra ã«æž¡ããã®ã¯ä»¥äžã§ãã
- ã²ãŒã ã®ã«ãŒã«
- ã²ãŒã ã®ãœãŒã¹ã³ãŒã
- ãã¹ãŠã®éã¿ããŒãã®ç©ºã®ãããã¯ãŒã¯
åŠç¿ããŒã¿ã¯ãªããããŒã«ããªãããããã¯ãŒã¯ãã²ãŒã ã§ãã¹ãããããšããäžå¯ãAstra 㯠1 åã®è¿ä¿¡ã®äžã§å
šéã¿ãæã¡èŸŒã¿ãç ç©¶è
ããã®çµæããã¹ãããŸããã
ããšãã° Breakout ã§ã¯ããã©ã³ã¯æ¬¡ã®ãããªãã®ã§ããã
- 第 1 å±€: ããŒã«ãã©ãã«èœã¡ããïŒå£ã§ã®åå°ãå«ãïŒãèšç®ããã
- 第 2 å±€: ããã«ããã®äœçœ®ã«åããã
| ã²ãŒã | PPOïŒ1 åã¹ãããïŒ | PPOïŒ5 åã¹ãããïŒ | AstraïŒ1 åã®è¿ä¿¡ãåŠç¿ãªãïŒ |
|---|
| Breakout | 15.8 | 25.3 | 517.4 |
| Seaquest | 5.6 | 88.9 | 53.1 |
| Asterix | 25.8 | 55.2 | 53.2 |
ã芧ã®éããAstra ã®ãããã¯ãŒã¯ã¯3 ã²ãŒã ãã¹ãŠã§1 åã¹ãããåŠç¿ã®ãããã¯ãŒã¯ãäžåããŸããã5 åã¹ãããåŠç¿ã®ãããã¯ãŒã¯ãšæ¯ã¹ãŠããBreakout ã§ã¯çŽ 20 åã®ã¹ã³ã¢ãAsterix ã§ã¯èèãSeaquest ã§ã¯çŽ 60% ã®ã¹ã³ã¢ã«éããŸããã
å²ç¢ãã¬ã€ã€ãŒã®æ§ç¯
äžã®ã²ãŒã çšãããã¯ãŒã¯ã¯ãAstra ã 1 åã®è¿ä¿¡ã§æã¡èŸŒããçšåºŠã®å°ããã§ãããå²ç¢ã§ã¯ã¯ããã«é£åºŠãäžããçŽ 150 äžåã®éã¿ãæã€ãããã¯ãŒã¯ãçšæããŸããããã㯠Astra ã®ããã³ããã«åãŸããããªã倧ããã§ãã
ããã§ä»£ããã«ãAstra ã« Python ã¯ãŒã¯ã¹ããŒã¹ãäžããéã¿ïŒåæç¶æ
ã¯ãã¹ãŠãŒãïŒãé²èЧã»ä¿åã§ããããã«ããŸãããããããã¯ãŒã¯ãå®è¡ããããšã¯äžå¯ãšããŸããããããã¯ãŒã¯ããã¹ããããå Žåã¯ãäžéã®ãããã¹ãå®è¡åæ°ïŒ0ã50ïŒãæ¶è²»ããªããã°ãªããŸãããåãã¹ãå®è¡ã§ã¯ 9Ã9 ç€ã§ 128 å±ããã¬ã€ããŸãã
Astra ã¯ãã®ãããã¯ãŒã¯ããå²ç¢ã®ããã°ã©ã ã®ããã«æ§ç¯ããŸããã
- ååã®å±€ã§åãã¬ã€ã€ãŒã®ç³ã®äœçœ®ãããŒãã³ã°ã
- äžéå±€ã§é£çµããç³ã®éããã©ããå±éºãªã°ã«ãŒããç¹å®ã
- æåŸã®å±€ã§ååæ³æã«ã¹ã³ã¢ãäžãããç³åããèªé£ä¿å
šã¯å ç¹ãæªæã¯æžç¹ã
ããã¯ã³ã³ãã€ã©ã«ãšãŠã䌌ãŠãããšæãããããããŸãããããã®éãã§ããDeepMind ã®Tracr ãè¡ããããã°ã©ã ãæ£ç¢ºãªãã©ã³ã¹ãã©ãŒããŒã®éã¿ã«èœãšã蟌ãããšãšéåžžã«è¿ãã®ã§ããéãã¯ãTracr ã¯å°çšèšèªã§æžãããããã°ã©ã ãå¿
èŠãšããäžæ¹ãAstra ã¯èªç¶èšèªã®ç®æšããããã°ã©ã èªäœãèæ¡ããç¹ã§ãã
ãã¹ãã倧ããªéããçãçç±
ãããŸã§èªãã§ãå®éšããšã«ã«ãŒã«ãéã£ãŠããããšã«ãæ°ã¥ãã§ããããã²ãŒã èå¥ã»å³åœ¢ã»è»ã»MinAtar ã®å®éšã§ã¯ Astra ã¯ããŒã«ãäžåãªãã§ããããèšèªã»å²ç¢ã»è¿åçã¢ãã«ã»Qwen ã®å®éšã§ã¯ Python ã䜿ããæã«ãããã¯ãŒã¯ã®ãã¹ããå¯èœã§ããã
ç§ã®èŠç«ãŠã§ã¯ãããã¯æçš¿å
šäœã§èŠéããããã¡ãªç¹ã§ããããã¹ãåæ°ã倧ãã圱é¿ããŠããããã§ãã
- å²ç¢: ãã¹ã 0 åã§ããAstra ã®ãããã¯ãŒã¯ã¯ããå°ããªåŠç¿æžã¿ 2 ã¢ãã«ã«å¯ŸããŠåç 5 å²ãè¶
ããŸããã50 åã§ã¯ãåèŠæš¡ã§2,000 äžå±ããåŠç¿ããã¢ãã«ã«åã¡ãŸããã
- Qwen: 10 åã 25 åã®ãã¹ãã§ã¯ã«ãŒããã»ãŒæ®ãã倧ããªé£èºã¯ 50 åã§ã®ã¿èµ·ããŸãããåçºã®å ±åãªã®ã§ããã®åŸåã¯ç€ºåçãšåãæ¢ããŠãã ããã
- è¿åçã¢ãã«ïŒãã¹ãåæ°ã§ã¯ãªãè¿œå æãããã§ãã䌌ã广ïŒ: éã¿ã ãã§ã¯å
šæã§ããããã¢ãã³ã·ã§ã³ã®æãããã 1 ã€å ãããš 3 çµãã¹ãŠæ£è§£ããŸããã
ãããã£ãŠãããŒã«ãªãã®çµæã¯æãé©ãã¹ããã®ã§ãããå®éšå士ãå
¬å¹³ã«æšªæ¯èŒã¯ã§ããªããšããããšã§ããããŸãã
GPT-6 Astra ã¯å®éã«ã©ããã£ãŠããã®ãïŒ
å
šãã¯ããã¯ã«å
±éãããã¿ãŒã³ããããŸããAstra ã¯äººéã®ãšã³ãžãã¢ã®ããã«æ¯ãèããŸãã
- ããã°ã©ã ãèšèšããã ãŸãåå±€ãäœããããïŒäŸïŒãããŒã«ã®èœäžäœçœ®ãæ±ãããïŒã決ããã
- èšèšãæ°å€ã«èœãšãã ãã®åœ¹å²ãæããããéã¿ãéžã¶ã
- æå°éã®å€æŽã«ãšã©ããã ãããã¯ãŒã¯ã®ä¿®æ£ã§ã¯ãè»ã®ã·ã§ãŒãã«ããã Qwen ã®é ãã¹ã©ã€ããŒã®ããã«ãå°ããªéšåã®è¿œå ã»åé€ã§æžãŸããã
- ïŒèš±ãããã°ïŒãã¹ãããã ãã¹ãå®è¡ãããå Žåã¯å€±æã®ä»æ¹ã芳å¯ãã圹ç«ã€å€æŽã ããæ®ãã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã° vs. ãã¡ã€ã³ãã¥ãŒãã³ã°ïŒã¢ãã«ç·šé
ãã¡ã€ã³ãã¥ãŒãã³ã°ãä»ã®ç·šéææ³ãšäœãéãã®ãããšæããããããããŸããã以äžã®è¡šã¯æãè¿ãææ³ãæ¯èŒããããããã 5 ã€ã®ãã¯ããã¯ã®ã©ãã«è¿ããã瀺ããŠããŸãã
| ææ³ | éã¿ã®å€å | åŠç¿ããŒã¿ã¯å¿
èŠïŒ | çšé | è¿ããã¯ãã㯠|
|---|
| ãã¡ã€ã³ãã¥ãŒãã³ã° / LoRA | åŠç¿ã¢ã«ãŽãªãºã ã倿°ã®éã¿ãèª¿æŽ | å¿
èŠ | æ°ããã¿ã¹ã¯ãæäœã®ä»äž | ãªã |
| ROME | å°ããæšçåããã 1 åã®æ°çæŽæ° | å°æ°ã®äŸ | ã¢ãã«ãç¥ãåäžã®äºå®ã®å€æŽ | ãã¯ãã㯠3 |
| ã¿ã¹ã¯æŒç®ïŒTask arithmeticïŒ | ãã¹ãã«ãéã¿ã®å æž | å¿
èŠ | 埮調æŽã¹ãã«ã®åæã»é€å» | ãã¯ãã㯠4 |
| Abliteration | åäžæ¹åã®åé€ | æ°çŸããã³ãã | æåŠãªã©ã®åäžæåã®é€å» | ãã¯ãã㯠4 |
| Weight Oracles | å€åãªãïŒèªãã ãïŒ | å¿
èŠïŒèšç·Žã®ããïŒ | éã¿ããé ããåé¡ã®æ€åº | ãã¯ãã㯠1ã»2 |
| ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã° | LLM ãæšè«ãç·šéãã | äžèŠ | èªç¶èšèªã®ç®æšããéã¿ãèªãã»çŽãã»æžã | 5 ã€ãã¹ãŠ |
ããã€ãè£è¶³ããŸãã
- ROME 㯠2022 幎ã«ããã¹ããŒã¹ããŒãã«ã¯ã·ã¢ãã«ã«ãããã®ãããªäºå®ãç¹å®ã®å±€ã«ä¿åãããŠãããå°ããªæŽæ° 1 åã§å€æŽã§ããããšã瀺ããŸããã
- ã¿ã¹ã¯æŒç®ã¯ãåŸ®èª¿æŽæžã¿ã¢ãã«ããå
ã¢ãã«ãåŒããšãã¹ãã«ãéã¿ãæ®ãããããä»ã¢ãã«ã«å ãããå€ãããã§ãããšç€ºããŸããã
- Weight Oracles ã¯æãè¿çžã§ã2026 幎 10 æã®è«æã§ããä»ãããã¯ãŒã¯ã®éã¿ãèªãã§ããã¯ãã¢ãªã©ã®åé¡ãèŠã€ããããã«ç¹ååŠç¿ããèšèªã¢ãã«ããç©å
·ãã©ã³ã¹ãã©ãŒããŒã§æŠå¿µå®èšŒããŸããã
倧ããªéãã¯ãWeight Oracles ããã®ä»äºå°çšã«èšç·Žãããã®ã«å¯ŸãããŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯æ±çšã® LLM ã䜿ãç¹ã§ãã
ççŽã«èšãã°ãæ°ããã®ã¯ãéã¿ãç·šéã§ãããããšèªäœã§ã¯ãããŸãããããã¯äœå¹Žãåããç¥ãããŠããŸãããæ°ããã®ã¯ãã²ãšã€ã®æ±çšã¢ãã«ãæšè«ã ãã§ããã倿§ãªäœæ¥ããã£ãŠã®ããããšã§ãã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã®éç
çµæã¯åºæ¿çã§ãããæ
éã«èªãããšãéèŠã ãšèããŸããèè
ãèªèº«ããããåæã®çµæããšè¿°ã¹ãŠããŸããäž»ãªæžå¿µç¹ã¯æ¬¡ã®éãã§ãã
1. æ¹åã§ã¯ãªãä¿®æ£ã§ãã
Gemma ã®å¯Ÿç
§å®éšã瀺ãéããQwen ã®å€§å¹
åäžã®å€§åã¯ã«ãŒã忢ã«ãããã®ã§ããèè
ãã®ãã°ãåæãã XenoSpectrum ã®å ±åã§ã¯ãç·šéåŸã«æ°ãã«è§£ãã 579 ä»¶ã®ç·Žç¿åé¡ã®ãã¡ã576 ä»¶ã¯ä»¥åã¯ããŒã¯ã³äžéã§æã¡åãããŠããŸããã
2. æ€èšŒããããã³ãã£ã¢ LLM 㯠1 ã€ã ã
æçš¿ã®å
šå®éšã§ GPT-6 Astra ã䜿ãããŸãããSimon Willison ã¯ããã®åŸãªãŒãã³ãŠã§ã€ãã® Qwen 3.8 27B ãçšããããŒã PC äžã§ã²ãŒã èå¥ã¿ã¹ã¯ãåçŸããŸãããã1 ã€ã®ã²ãŒã ã¯å€ããŸãããä»ã®ããã³ãã£ã¢ã¢ãã«ãä¿®çãæžã蟌ã¿å®éšã§ Astra ã«äžŠã¹ããã¯ãèè
ããèšç»ãããã³ãããŒã¯ãçããã§ãããã
3. æ»èªãåããŠããªã
ããã¯è«æã§ã¯ãªãããã°èšäºã§ããèè
ãã¯å®å
šãªãã°ãå
¬éããŠãããHugging Face ã® Ahsen Khaliq ã¯ç·šéæžã¿ Qwen ã¢ãã«ãåæ§ç¯ããŠè©äŸ¡ãåå®è¡ããŸãããèè
ã«ããã° MATH-500 㯠51.0%ãããŒã¯ã³äžéå°é㯠31.6% ã§ãããããã¯å
¬éãã§ãã¯ãã€ã³ãã®æçšãªæ€èšŒã§ãããAstra èªèº«ãç·šéãè¡ãéçšã®ç¬ç«åçŸãšã¯ç°ãªããŸãã
4. å ±åçµæãæå©å¯ãã®å¯èœæ§
æ¹è©å®¶ã¯ãåæã®ããã³ãã詊è¡ãå ±åçµæã«å«ãŸããŠããªãç¹ããAstra ã®æšè«åŽåãå®éšéã§ã°ãã€ããŠããç¹ãææããŠããŸããããã³ããã®èšãåããå°ãå€ãããšãã®æåºŠè©ŠéšãèŠåœãããŸããã§ããã
5. ã³ã¹ããäžæ
æè¯ã® Qwen å®è¡ã¯ 158 ã¿ãŒã³ã20.3 æéãèŠãããã®ãã¡ 4.7 æéã¯ãµãŒãã¹åæ¢ã«ãããã¹ã§ãããéåžžã®ãã¡ã€ã³ãã¥ãŒãã³ã°ãšã®æ¯èŒããªããããæéãã³ã¹ãã®ç¯çŽã«ãªããã¯çŸæç¹ã§ã¯äžæã§ãã
6. æ¯èŒãããå¯å®¹
ã²ãŒã çµæã®æ¯èŒå¯Ÿè±¡ã¯åŠç¿æžã¿ãããã¯ãŒã¯ã§ãããåçŽãªææžãã³ãŒããšã®æ¯èŒã¯ãããŸããããããã¯åŽ©ãã§ãããŒã«ã远ãã ããã®ãã¬ã€ã€ãŒãæžãã®ã¯é£ãããªããããåŠç¿æžã¿ãããã¯ãŒã¯ã«åã£ããšãã£ãŠãæåã®å°è±¡ã»ã©é©ãã§ã¯ãªããããããŸãããèè
ãã¯ãåããŒã¹ã©ã€ã³ãããããã®å®éšã®çã䞻匵ïŒçµéšãŒãã§ã©ããŸã§è¡ããããªã©ïŒã ããæ€èšŒããŠãããšå¿çããŠããŸãã
èè
ãã¯ãè¿åçã¢ãã«ã Gemma ã®çµæãªã©ããŸããããªãã£ãç¹ãççŽã«è¿°ã¹ãŠããããã®ç¹ã¯è©äŸ¡ããŸãããã ãèŠåºãã«ãªãæ°å€ã¯ãäžè¬åã®èšŒæãšããããææãªåæãµã€ã³ãšããŠåãæ¢ããã¹ãã§ãããã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã®å¿çš
ãããã®å¶çŽããã£ãŠããä»åŸããã«é²å±ããã°ã©ããžåããããæ³åãããšããããããŸããèè
ããæããå¯èœãªå¿çšäŸã¯æ¬¡ã®éãã§ãã
| å¿çš | æ³å®ãããå§¿ |
|---|
| ãã¹ããªãã§ã®ã¢ãã«æ€æ» | éã¿ãèªãã ãã§ãããã¯ãã¢ã®ãããªé ããåé¡ãã¢ãã«å
¬éåã«çºèŠããã |
| äœã³ã¹ããªã¢ãã«æ§ç¯ | 倧éããŒã¿åéãé·æéåŠç¿ãªãã«ãè¯ããããã¯ãŒã¯ãäžåºŠã«èšå®ããã |
| ããŒã¿ãªãã§ã®ã¹ãã«è¿œå ã»åé€ | LLM ã«ã¢ãã«ããç¹å®ã¹ãã«ã®åé€ãçŽæ¥äŸé Œããã倿Žã¯å°æ°ã®éã¿ã«éãããããããã以å€ã¯ä¿ããã埮調æŽã§ãããã¡ãªå¿åŽãé¿ããããã |
| ç°ãªãã¢ãã«éã®ããã¯ã¹ | æšè«ã¢ãžã¥ãŒã«ã®ãããªæçšéšåããèšèšã®å€§ããç°ãªãå¥ã¢ãã«ãžç§»æ€ãããçŸåšã®ããŒãžææ³ã¯åäžèšèšéã§ããæ©èœããªãã |
| AI ã®èªå·±æ¹å | åŠç¿ãžã§ããèµ·åããã³ãŒããæžãã®ã§ã¯ãªããèªåèªèº«ã®éã¿ãèªã¿ãæ¹åããã¢ãã«ã |
æåŸã®ç¹ããããã®ç ç©¶ã«èª°ããååãã§ã¯ãªãçç±ã§ããPauseAI ã® CEOãMaxime Fournes ã¯ãã®ç ç©¶æ¹åãç¡è¬ãšåŒã³ãŸãããå®éã®é²å±ãå±éºãª AI éçºãå éãåŸãããã§ãããã ãããã®æçš¿ã¯ãããªãèªå·±æ¹åã瀺ããŠã¯ããªãããšã¯ææããŠãããŸãã
ããååããªå®å
šé¢ããããŸããLLM ãè¿åæ§ãããã¯ãã¢ã®ãããªäºè±¡ãéã¿ã®èªã¿åãã ãã§ç¢ºå®ã«çºèŠã§ããããã«ãªãã°ãå
¬éåã®ã¢ãã«æ€æ»ã«æçšãªææ®µãšãªãããŸãã
ä»åŸã¯ïŒ
èè
ãã¯çŸåšãåçš® LLM ã®å®åãæž¬ããã³ãããŒã¯ãšããããæ¯æŽããããŒãã¹ãæ§ç¯äžã ãšè¿°ã¹ãŠããŸããä»åŸåãçµãåããšããŠã¯ã次ã®ãããªãã®ããããŸãã
- ä»ã® LLM ã«ãå¯èœãªã®ãããããšã Astra åºæãªã®ãïŒ
- ã«ãŒãã®ãããªåé¡é€å»ã ãã§ãªããæ¬åœã«æ°ããã¹ãã«ã远å ã§ããã®ãïŒ
- éã¿ãäœååãå€ãã¯ããã«å€§ããªã¢ãã«ã§ãæ©èœããã®ãïŒ
- éåžžã®ãã¡ã€ã³ãã¥ãŒãã³ã°ããå®äŸ¡ã«ãªãããšã¯ããã®ãïŒ
å
¬é以åŸãé²å±ã¯éãããã§ãã11 æ 12 æ¥ã®æŽæ°ã§ã¯ãããããæ¹çãéååè·¯ãžã®æ¡åŒµãå«ã 100 ä»¶è¶
ã®è¿œè©Šã»æ¡åŒµãååšããIWIBench ãšãããã³ãããŒã¯ãç»å Žãããšå ±åãããŠããŸãã
ãèªèº«ã§ç¢ºããããå Žåã¯ãGitHubã§å
šå®éšã®å®å
šãªãã°ïŒAstra ã«éã£ãå
šããã³ãããšæšè«ã®èŠçŽãå«ãïŒãé²èЧã§ããŸããä¿®æ£æžã¿ Qwen ã¢ãã«ã Hugging Face äžã«ããããã©ãŠã¶ã§è©Šããã©ã€ããã¢ããå
èšäºãããªã³ã¯ãããŠããŸãã
ãŸãšã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯ãåæçãªããè£ä»ããåºãŠããæ°ããã¢ã€ãã¢ã§ããæ±çš LLM ããåŠç¿ãªãã§å°ããªãããã¯ãŒã¯ã®æ©èœãèªã¿åããç«ã¡åŸçããè»ãã«ãŒãããèšèªã¢ãã«ãçŽããããã«ã¯ãŒãããã²ãŒã çšãããã¯ãŒã¯ãæ§ç¯ããŸããã
æãé©ããã®ã¯ããã®ããæ¹ã§ããè¬ãããããªãã¯ã§ã¯ãªãããããã¯ãŒã¯ã®ç«¯ãèªã¿ãäžã®ãµãŒããããæ¢ããã·ã§ãŒãã«ãããè¶³ããæ¹åãåããæ°å€åããåã«ããã°ã©ã ãèšèšããââãŸãã§ãšã³ãžãã¢ã®ããã§ããã
ãã ããã㯠1 ã€ã®ã¢ãã«ã«åºã¥ãåæå ±åã§ãããæå€§ã®ææã¯ã¢ãã«ãè³¢ããããšãããããã°ä¿®æ£ã«è¿ããã®ã§ããç¹ã¯æ¹ããŠåŒ·èª¿ããããšæããŸããä»åŸãé²å±ããã°ãæãæçšãªã®ã¯åŠç¿ã®ä»£æ¿ã§ã¯ãªããã¢ãã«ã®ç£æ»ããããã°ã«ãªãã§ãããã
ãã®çš®ã®ç ç©¶ã远ãçŽ å°ãæŽãããå Žåã¯ãLarge Language Models (LLMs) Concepts ã§ã¢ãã«ã®å
åŽã®ä»çµã¿ãåŠã¹ãŸãããšãŒãžã§ã³ãåŽã«é¢å¿ããããªããIntroduction to AI Agents ãè¯ã次ã®äžæ©ã§ãã
ããªãã®å¥œããªæ²ã®é³å£°ãã¡ã€ã«ããã²ãšç¶ãã®æ°å€ãªã¹ããšããŠçŽã«å°å·ããŠæž¡ããããšããŸããããªãã¯ãã®æ°åãèŠãã ãã§ããããã©ã®æ²ãèšãåœãŠãããã§ããããïŒ
æ²ã®æ å ±ã¯ç¢ºãã«ãã®äžã«ãããŸãããç§ãã¡ã«ãšã£ãŠã¯æå³ããªããªãäœåãã®å€ã®äžã«é ãããŠããŸãã
ãã¥ãŒã©ã«ãããã¯ãŒã¯ããŸã£ããåãåé¡ãæ±ããŠããŸãããããã¯ãŒã¯ãç¥ã£ãŠããããšã¯ãã¹ãŠéã¿ã«ä¿åãããŠããŸãããããã¯åãªãæ°åã§ãã
ãã£ãŒãã©ãŒãã³ã°ãèªçããŠä»¥æ¥ãç§ãã¡ã¯ãã®æ°åãèŠãŠããã®ãããã¯ãŒã¯ã¯ãããã¯åŽ©ãããã¬ã€ãããããã®ãããã¯ãŒã¯ã¯ç¹å®ã®ã¿ã¹ã¯ãããªãããšèªä¿¡ãæã£ãŠèšãããšãã§ããŸããã§ããããŸããç©å ·ã¬ãã«ä»¥äžã®ãããã¯ãŒã¯ã«ã€ããŠã¯ãèªåãã¡ã§ãã®æ°åãæžãããšãã§ããŸããã§ããããã®ä»äºã¯åžžã«åŠç¿ïŒãã¬ãŒãã³ã°ïŒã«å§ããããŠããŸããã
2026幎11æ2æ¥ãMIT ã³ã³ãã¥ãŒã¿ç§åŠã»äººå·¥ç¥èœç ç©¶æïŒMIT CSAILïŒã®ããŒã ã¯ãOpenAI ã®æå ç«¯å€§èŠæš¡èšèªã¢ãã«ïŒLLMïŒã§ãã GPT-6 Astra ããã®äž¡æ¹ãå®çŸã§ããããšã瀺ããŸãããAstra ã¯ãããã¯ãŒã¯ã®çã®éã¿ãèŠãŠãã®æ©èœãææ¡ããå£ããŠããç®æãèŠã€ããããã«ã¯ãŒãããæ°ãããããã¯ãŒã¯ãæžãäžããããšãŸã§ããã£ããåŠç¿ãªãã§è¡ããŸããèè ã® Rikiya TakehiãJayden TeohãAlex ZhangãOmar Khattab ã¯ããããŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ãšåŒã³ãå ã®ããã°èšäºã§æå±ããŸããã
æ¬èšäºã§ã¯ããããã©ã®ããã«æ©èœããã®ãã«çŠç¹ãåœãŠãŸããå®éšå¥ã§ã¯ãªããLLM ãçšããææ³å¥ã«æŽçããŠããŸããå®éšçµæãå ã«ç¥ãããå Žåã¯ãåœç€Ÿã®ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°æŠèŠãã芧ãã ããã
ãã¯ããã¯ã«å ¥ãåã« LLM ãšãã®åºç€ã§ãããã©ã³ã¹ãã©ãŒããŒã®ä»çµã¿ããããããããå Žåã¯ãIntroduction to LLMs in Python ã³ãŒã¹ã§åºç€ãæ±ã£ãŠããŸããéã¿ã®èªã¿æžããè¡ãã¢ãã«ãèŠãéã«åœ¹ç«ã¡ãŸãã
ã²ãšç®ã§ãããèŠç¹
æ¬èšäºã§æ±ãå å®¹ã¯æ¬¡ã®éãã§ãã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ãšã¯ïŒ
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ãšã¯ããã¥ãŒã©ã«ãããã®éã¿ãããã°ã©ã ã®ããã«æ±ãçºæ³ã§ããæ±çš LLM ããåéšãäœãããŠããããæšè«ããªããéã¿ãçŽæ¥èªã¿æžãããåŸé éäžã®ãããªåŠç¿ã¢ã«ãŽãªãºã ã«é Œããªããšãããã®ã§ãã
ããå¹³æã«èšãã°ãLLM ããããã¯ãŒã¯ãéããŠèªåã§æ°åãèŠãŠãçãããã£ãŠå€æŽããããšããããšã§ãã
ç§ã¯ããããç¬ãèšç·Žããããšãšããããç¬ã®é ç·ãçµã¿æ¿ããããšã®éãã ãšæããŠããŸããèšç·Žã¯å€ãã®äŸãèŠããŠåŸã ã«è¡åãå€ããããšãæåŸ ããã®ã«å¯Ÿããé ç·ã®çµã¿æ¿ãã¯å éšã«å ¥ããæ åœéšåãèŠã€ããŠçŽæ¥å€ããŸãã
ããã¯ãLLM ãããªãã®ããã«åŠç¿ã¹ã¯ãªãããæžãã®ãšã¯å¥ç©ã§ããLLM ãããã以åããã§ããŠãããšããŠããããã§ã¯åŠç¿ã¹ã¯ãªããèªäœãååšãããLLM ã¯å®éã®æ°å€ãã®ãã®ãæ±ããŸãã
èè ãã¯ããŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯åäžã®ææ³ã§ã¯ãªãæŠå¿µã§ãããåŸé ãæé€ãããã®ã§ã¯ãªããšåŒ·èª¿ããŸããä»åã®å®éšã§ã¯ãæšè«ã ãã§äœãã§ããããåãåããããããããŠåŸé ãçŠæ¢ããŸããã
èšäºã§ã¯ 3 ã€ã®æ¹æ³ã§ãã®ã¢ã€ãã¢ãæ€èšŒããŠããŸããå šäœåãçè§£ããã«ã¯ããããã¡ã°ãç°¡åã§ãããïŒå³ 1ïŒã
å³ 1: æ¬èšäºã§æ€èšŒãã 3 ã€ã®èœåãç»åã¯çè äœæã
ããããè¡ã£ã LLM 㯠GPT-6 Astra ã§ãOpenAI ã 2026 幎 9 æ 3 æ¥ã«å ¬éããŸãããããã€ãã®å®éšã§ã¯ããã¹ãŠã®ããŒã«ïŒãŠã§ãæ€çŽ¢ãã³ãŒãå®è¡ãå«ãïŒããªãã«ããæ°åãèªã¿ãèããããšã ããå¯èœãªç¶æ ã«ããŸããã
ããã 3 ã€ã®èœåã¯ããããç¬èªã®ãã¯ããã¯ã䜿ã£ãŠããã以éã§é ã«è§£èª¬ããŸãã
éåžžããã¥ãŒã©ã«ãããã®éã¿ã¯ã©ãäœãããïŒ
ãã¯ããã¯ã«å ¥ãåã«ãéã¿ãéåžžã©ã®ããã«äœãããããæŒãããŠãããšåœ¹ã«ç«ã¡ãŸããããã§ãªããšããã®ç ç©¶ããªãé©ããåŒãã ã®ããèŠãã«ããããã§ãã
åŠç¿ã®åŸ©ç¿
ãã¥ãŒã©ã«ãããã¯éã¿ïŒãã©ã¡ãŒã¿ïŒãšåŒã°ããå·šå€§ãªæ°ã®éåã§ããç»åãæãšãã£ãå ¥åãäžãããšããããã®æ°å€ãæãåããããè¶³ãåããããããŠåºåãçæããŸããããã¯ãã©ã¯ãŒããã¹ãšåŒã°ããŸãã
åŠç¿ãšã¯ããããã®æ°ã«å¯ŸããŠè¯ãå€ãéžã¶ããã»ã¹ã§ããæšæºçãªããæ¹ã¯åŸé éäžæ³ã§ããããã¯ãŒã¯ãèµ°ãããåºåã®èª€å·®ã枬ããåéã¿ãã©ã¡ãã«åããã°å°ããã·ã«ãªãããèšç®ãããã¹ãŠã®éã¿ãã»ãã®å°ããã€åãããŸãããããæ°çŸäžåãç¹°ãè¿ããããåŠç¿ã¯é«ã³ã¹ãã«ãªããŸãã
éã¿ãæžããã 4 ã€ã®æ¹æ³
ãããŸã§ã«äœ¿ã£ãããšã®ãããã¥ãŒã©ã«ãããã®éã¿ã¯ãå¿ ãäœãããã®æ¹æ³ã§æ±ºãŸã£ãŠããŸãããããŸã§ã¯ããã®ãäœããã¯æ¬¡ã® 4 ã€ã®ããããã§ããã
AdderBoard ã¯ç¹æ®ãªã±ãŒã¹ã§ãã10 æ¡ã®å ç®ãã§ããæå°ã®ãã©ã³ã¹ãã©ãŒããŒãäœãã³ãã¥ããã£ãã£ã¬ã³ãžã§ãåæ°çš®é¡æªæºã®éã¿å€ã§æäœãããããšã³ããªãŒããããŸãããšã¯ãããChatGPT ã®ãããªèŠæš¡ãšã¯ãŸã£ããå¥ç©ã§ãã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯ããã®ãªã¹ãã« 5 ã€ç®ã®éžæè¢ãã€ãŸãéã¿ã«ã€ããŠæšè«ããæ±çš LLMãå ããŸãã
å³ 2: éåžžã®åŠç¿ãšãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã®æ¯èŒãç»åã¯çè äœæã
ã§ã¯ããªãéã¿ãèªããªãã®ãïŒ
éã¿ããã ã®æ°åãªãããªãèªãã®ãé£ããã®ã§ãããããäž»ãªçç±ã¯ 2 ã€ãããŸãã
第äžã«ããããã¯ãŒã¯ã¯ 1 ã€ã®æŠå¿µã 1 ã«æã«ä¿åããŸãããã²ãšã€ã®æŠå¿µã¯éåžžãå€ãã®ãã¥ãŒãã³ã«åæ£ããŠããã1 ã€ã®ãã¥ãŒãã³ã¯åæã«å€æ§ãªæŠå¿µã«è²¢ç®ããŸãã
第äºã«ãåããããã¯ãŒã¯ã§ã倿§ãªè¡šçŸãå¯èœã§ããããå±€ã®ãã¥ãŒãã³ã®é åºãå ¥ãæ¿ããæ¬¡ã®å±€ãããã«åãããã°ããããã¯ãŒã¯ã®æåã¯ãŸã£ããåãã®ãŸãŸã§ããã€ãŸããåãä»äºããã 2 ã€ã®ãããã¯ãŒã¯ã§ããéã¿ã¯èŠãç®ã倧ããç°ãªãåŸãŸãã
以äžã®åãã¯ããã¯ã¯ãåºæ¬çã«ã¯ããã 2 ã€ã®åé¡ãåé¿ããæ¹æ³ã ãšèããŠãã ããã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã倧ããªäžæ©ãšèšãããçç±
ãã®æçš¿ãæ³šç®ãéããã®ã¯ããã倧ããªæœ®æµã®å»¶é·ã«ããããã§ããããæ°å¹Žã§ãç§ãã¡ã¯ LLM ã«èªåŸçã«æ¹åããã察象ãã©ãã©ãåºããŠããŸããã
å³ 3: æéãšãšãã« LLM ã«æ¹åãããŠãã察象ãç»åã¯çè äœæã
éèŠãªã®ã¯ããããã¯ãã¹ãŠããã¹ããã€ãŸã人éãæã§ç·šéã§ããããã³ããã Python ãã¡ã€ã«ã ã£ãããšã§ããèè ãã¯ãããã¹ãã LLM ã«äœ¿ã£ãŠããã®ã¯ããããæ¹åããããè¯ãææ®µããªãã£ãããã ãšææããŸããäžæ¹ãéã¿ã«ã¯åŸé éäžããã£ããããèª°ãæ¬æ°ã§ LLM ã«é Œãããšã¯èããŸããã§ããã
æ¬æçš¿ã¯ãåŠç¿ã¹ã¯ãªãããå®å šã«é£ã°ããLLM ã«éã¿ãã®ãã®ãå€ãããããã©ããªãããåããŠããŸãã
ã¹ãã« 1: LLM ã¯ã©ããã£ãŠéã¿ãèªããïŒãã¯ãã㯠1 ãš 2ïŒ
ç ç©¶è ãã¡ã¯ããŸãéã¿ããã¹ãŠãã¬ãŒã³ããã¹ããšã㊠Astra ã®ããã³ããã«è²Œãä»ããããã»ã©å°ããªãããã¯ãŒã¯ããå§ãããã®åŸãAstra ã Python ã§è§£æãããã倧ããªã¢ãã«ãžãšç§»ããŸãããAstra ã¯äž»ã« 2 ã€ã®ãã¯ããã¯ã§èªã¿åããŸããã
ãã¯ãã㯠1: å ¥åºåã®éã¿ããæ§é ãèªã
ç§ã®èãã§ã¯ããããã¯ãŒã¯ãèªãåºçºç¹ãšããŠæãç°¡åãªã®ã¯ãã®ç«¯ïŒãšããžïŒã§ããæåã®å±€ã¯å ¥åïŒç»é¢ã®ãã¯ã»ã«ãªã©ïŒã«çŽæ¥ã€ãªãããæåŸã®å±€ã¯åºåïŒãã¬ã€ã€ãŒãã§ããè¡åãªã©ïŒã«çŽæ¥ã€ãªãããŸããã©ã®æ°ãäœã«æ¥ç¶ããŠããããããã£ãŠããã®ã¯ãã® 2 å±€ã ãã§ãã
ãã®ãããã¯ãŒã¯ã¯ã©ã®ã²ãŒã ããã¬ã€ããŠããïŒ
ç ç©¶è ã¯ã匷ååŠç¿ã¢ã«ãŽãªãºã ã® Proximal Policy OptimizationïŒPPOïŒ ã§ããããã¯åŽ©ããPongãPac-ManãFreeway ã®ç°¡ç¥åãã 10Ã10 ã°ãªããçããã¬ã€ãã 4 ã€ã®å°ããªãããã¯ãŒã¯ãåŠç¿ãããŸããã
ãã®èšå®ã®å·§åŠãªç¹ã¯ã4 ã€ã®ãããã¯ãŒã¯ããã¹ãŠåããµã€ãºïŒå 17,253 åã®éã¿ïŒã§ãåãå ¥åãåãåããåãè¡åãå¯èœãªããšã§ããã€ãŸããèŠåããå¯äžã®æ¹æ³ã¯æ°å€ãèªãããšã§ããã
åãããã¯ãŒã¯ã®éã¿ã Astra ã«è²Œãä»ããéžæè¢ãªã¹ããäžããã«ããã®ãããã¯ãŒã¯ã¯ã©ã®ã²ãŒã ã§åŠç¿ãããïŒããšå°ããŸãããAstra ã¯ããã§ãå š 4 ã€ã§æ£ããã²ãŒã åãæããŸããã
å³ 4: Astra ãçã®éã¿ãèªã¿åãããããã¯åŽ©ããç¹å®ãç»åã¯çè äœæã
å°è±¡çã ã£ãã®ã¯ããã®çµè«ã«ã©ããã©ãçãããã§ãããããã¯åŽ©ãã®ãããã¯ãŒã¯ã§ã¯æ¬¡ã®ããã«æšè«ããŸããã
ããããçµã¿åããããšãããã¯åŽ©ãã«ãªããŸããæåã® 2 ã€ã®æãããã¯å ¥ååŽã®éã¿ãããæåŸã¯åºååŽã®éã¿ããåŸãããŠããããŸãã«ãã®ãã¯ããã¯ã®èŠç¹ã§ãã
ãã®ã¢ãã«ã¯ã©ããªå³åœ¢ãæãïŒ
次ã«ç ç©¶è ãã¡ã¯ãå°ããªç»åçæã¢ãã«ïŒæ¡æ£ã¢ãã«ãå€ãã®ç»åçæ AI ã®ç³»èïŒããåã»äžè§åœ¢ã»åè§åœ¢ãæãããã«åŠç¿ãããŸãããããã§ã Astra ã¯éã¿ã ããåãåããããŒã«ãã¢ãã«å®è¡ææ®µãäžããããŸããã§ããããåã¢ãã«ã«ã€ããŠæ£ããå³åœ¢åãçããŸããã
ã²ãšã€å¥åŠãªèª€ãããããŸãããå®éã®ã¢ãã«ã¯å¡ãã€ã¶ãã®å³åœ¢ãæãã®ã«ãAstra ã¯ãã¹ãŠèŒªéãšããŠèšè¿°ããã®ã§ããç ç©¶è ãã¡ã¯ããã¯è¯ãå åã ãšäž»åŒµããŸããAstra ãéã¿ã«ä¿åãããå³åœ¢ã®ãçžããèªãã§ããããšã瀺åããé ã®äžã§ãã£ããã¢ãã«å šäœãå®è¡ããŠããããã§ã¯ãªãããã§ãã
ãã¯ãã㯠2: äžéå±€ã®ãµãŒããããèŠã€ãã
æåã® 2 ã€ã®å®éšã§ã¯ãAstra ã¯ãããã¯ãŒã¯ã®æåãæåŸã®å±€ã ããèªãã°æžã¿ãŸããããããå®éã®èšèªã¢ãã«ã§ã¯ãæèã®å€ããäžéå±€ã§èµ·ãããŸãã
ããã§ 2 ã€ç®ã®ãã¯ããã¯ãã€ãŸããµãŒãããæ¢ããç»å ŽããŸãããµãŒããããšã¯ããããã¯ãŒã¯å éšã§ç¹å®ã®ä»äºãåæ ããŠè¡ãå°ããªéšå矀ã§ããäœãããŠãããããããã°ãå ¥åãäœããç¥ããªããŠããããã¯ãŒã¯ã«ã€ããŠå€ããåŠã¹ãŸãã
ãã®ã¢ãã«ã¯ã©ã®èšèªãåŠãã ïŒ
ç ç©¶è ãã¡ã¯å°ããªèšèªã¢ãã«ïŒããããçŽ 720 äžåã®éã¿ïŒã 3 ã€åŠç¿ãããŸããã
ãã®åŸãåã¢ãã«ã®æåãšæåŸã®å±€ãåãèœãšããäžéããã㯠4 ã€ã ããæ®ããŸããããããã®å±€ããªããã°ãã©ã®æ°ãã©ã®æåã«å¯Ÿå¿ãããã¯éã¿ããã¯ãããããããã¹ãäžã§ã¢ãã«ãèµ°ãããããšãã§ããŸãããã€ãŸããã¯ãã㯠1 ã¯äœ¿ããŸããã
ä»å㯠Astra ã« Python ã§ã®æ°å€è§£æãèš±å¯ããŸããããã¢ãã«ãå®è¡ããããšã¯çŠããŸãããããã§ã 2 åã®è©Šè¡ã§ãããã3 ã¢ãã«ã®èšèªãæ£ããç¹å®ããååã®ç¢ºä¿¡åºŠã¯ 50ã60% çšåºŠã§ããã
ã§ã¯ Python ãã©ãèŠæããã®ããAstra ã¯ã€ã³ãã¯ã·ã§ã³ããããæ¢ããŸããã
ã€ã³ãã¯ã·ã§ã³ããããšã¯ããã©ã³ã¹ãã©ãŒããŒå ã§ãããã¹ãäžã«ä»¥åçŸãããã¿ãŒã³ãèŠã€ããããã«ç¶ããã®ãã³ããŒããéšåã§ããããšãã°ãtotãããtotalãã«è£å®ããã®ã¯ãå ã«ãtotalããæžãããŠããããã§ããã³ãŒãã§ã¯å€æ°åã®ç¹°ãè¿ããéåžžã®æç« ããã¯ããã«å€ããPython ã¢ãã«ã¯ãããå€ãå¿ èŠãšããŸããAstra ã¯è©²åœãããã®ããã¹ãŠèŠã€ãã誀èªã¯ããŸããã§ããã
Astra ã¯ãç°ãªããããã®éã¿è¡åãæãåãããã©ã®ãããå士ããäŒè©±ãããŠããããèŠãããšã§ç¹å®ããŸãããããã¯ç ç©¶è ãèšåããŠããªã 2021 幎ã®è«æ A Mathematical Framework for Transformer Circuits ã®ææ³ã«äžèŽããŸãã
ããŒãåã«ã€ããŠã¯ãã¢ãã«ã远跡ãã¹ãæåãå°ãªãããã«èŠãããšææããŸãããå®éãããŒãåã§ã¯ qãlãvãx ãšãã£ãæåã¯äœ¿ããŸããã
èªã¿åããç Žç¶»ãããšããïŒã©ã¡ããè¿åçã¢ãã«ãïŒ
ããã§ 2 ã€ã®ãã¯ããã¯ãå¹ããªããªããŸããããã¯ãæåã«ããŸããããªãã£ãå¯äžã®å®éšã§ãã
ç ç©¶è ãã¡ã¯ Google ã®Gemma 3 270M ã 2 ã€çšæãããŸã£ããåãæ¹æ³ã§ãã¡ã€ã³ãã¥ãŒãã³ã°ããŸãããã1 ç¹ã ãéããèšããŸãããçæ¹ã¯æ£ããçããè¿ãããã«åŠç¿ããããçæ¹ã¯è¿åçïŒãŠãŒã¶ãŒã®ç€ºåã«åŸã£ãçããåžžã«è¿ãïŒã«ãªãããã«åŠç¿ããŸãããããã«å ¥åå±€ãšåºåå±€ãåºå®ããAstra ããããèŠãã·ã§ãŒãã«ãããå°ããŸããã以äžã¯äŸã§ãã
éã¿ã ãããã¯ãAstra 㯠4 åã®è©Šè¡ãã¹ãŠã§ã©ã¡ããè¿åçã¢ãã«ããèŠåããããŸããã§ãããäž¡ã¢ãã«ã®çžéç¹ã¯å€æ°èŠã€ããŸããããæåã«çµã³ã€ããããšãã§ããªãã£ãã®ã§ãã
ããã§ç ç©¶è ã¯æãããã 1 ã€ã ã远å ããã¢ãã«ã Astra èªèº«ã®æžããããã¹ããèªãéã«ãåã¢ãã³ã·ã§ã³ããããã©ããèŠãŠããããèŠããŸããïŒãã ãå®éã®åºåã¯èŠããªãïŒãããã«ãããAstra ã¯3 çµãã¹ãŠã§è¿åçã¢ãã«ãæ£ããç¹å®ããŸããã
ããã«éãã®æåšãçªãæ¢ããŸãããã¢ãã³ã·ã§ã³ããã 10.2 ã§ã¯ãèª å®ãªã¢ãã«ã泚æã®94.3%ã質åäžã®æ°å€ã«å²ããã®ã«å¯Ÿããè¿åçã¢ãã«ã¯85.7%ããŠãŒã¶ãŒã®ç€ºåããçãã«å²ããŠããŸããã
ããã¯æ¬èšäºã§æãéèŠãªçµæã ãšå人çã«èããŸããæå 端㮠GPT-6 Astra ã§ãã£ãŠããéã¿ã ãããæåãèªã¿åãã®ã¯ãå®éã®èšèªã¢ãã«ã§ã¯äŸç¶ãšããŠéåžžã«é£ããããšã瀺ããŠããŸãã
ã¹ãã« 2: LLM ã¯ã©ããã£ãŠéã¿ãçŽããïŒãã¯ãã㯠3 ãš 4ïŒ
éã¿ãèªãã ãã§ãååã«å°è±¡çã§ãããçŽãããšã¯ãããã倧ããé£ããäžæ©ã§ããAstra ã¯ãããã¯ãŒã¯ãäœãããŠããããçè§£ããäœããã¹ãããå®ããå·®ãåããããã«é©åãªæ°å€ã倿Žããªããã°ãªããŸãããããã§ã¯ç°ãªã 2 ã€ã®ãã¯ããã¯ãçšããŸããã
ãã¯ãã㯠3: å°ããªã·ã§ãŒãã«ããåè·¯ã远å ãã
ãã®æ¹æ³ã§ã¯ãããã¯ãŒã¯ã®å€§éšåã«ã¯æãä»ãããå ã®ãããã¯ãŒã¯ã倱æããç¶æ³ã§ã®ã¿äœåããå°ããªåè·¯ã远å ããŸãã
ãã€ãç«ã¡åŸçããè»
ãã®å®éšã§ã¯ãç ç©¶è 㯠2D ç©çã²ãŒã ã®Kinetixã䜿ããPPO ã§è»ãåéãç»ãããã«åŠç¿ãããŸãããè»ã¯ååãªå©èµ°ãã€ããããã«ãŸãåŸéãããã®åŸã«åé²å éããå¿ èŠããããŸãã
ãããåŠç¿æžã¿ãããã¯ãŒã¯ã¯ãããçè§£ã§ããŸããã§ãããæ¯åãŸã£ããåã«çªã£èŸŒã¿ãå¿ ãç«ã¡åŸçããŠããŸãã®ã§ãã
ããã§ç ç©¶è 㯠Astra ã«æ¬¡ã® 3 ã€ãæž¡ããŸããã
Astra ã¯äžåºŠã®è©Šè¡ã§ãããŒã«ãªãã®ãŸãŸè»ã倱æããçç±ã説æããæ°ããéã¿ãæžããŸããã倿ŽåŸãè»ã¯1,024 ãšããœãŒããã¹ãŠã§æåãïŒ0 ããã®æ¹åïŒã倿Žããéã¿ã¯å šäœã®ããã3.5%ã§ããã
Astra ã¯ãããã¯ãŒã¯å šäœãæžãæããã®ã§ã¯ãªããåå±€ã®æ°åã®ãã¥ãŒãã³ãéãå°ããªã·ã§ãŒãã«ããã远å ããŸãããããã¯éå§æã ãäœåããŠè»ãåŸéãããåé²ãžåãæ¿ããŸããè»ãèµ€ã垯ãè¶ãããšã·ã§ãŒãã«ããã¯èªåã§ãªãã«ãªãã以éã¯å ã®ãããã¯ãŒã¯ãéè»¢ãæ ããŸãã
ããã¯æå®ã飿ã ããã³ãã«ãåã£ãŠãããçåŸã«æ»ããããªãã®ã ãšèãããšããããããã§ããããå ã®ãããã¯ãŒã¯ãå®å šã«å£ããŠããããã§ã¯ãªããé転ã¯ã§ãããç¹å®ã®é£æã ãèŠæã ã£ããããä»ã¯ãã®ãŸãŸã«ããã®ãçã«ããªã£ãŠããŸãã
ãã¯ãã㯠4: ããæ¹åãèŠã€ããŠåãé€ã
ãã®æ¹æ³ã¯ãæåãã¢ãã«å éšã® 1 ã€ã®æ¹åã«æ²¿ã£ãŠçŸãããšããèãã«åºã¥ããŸãããã®æ¹åæåãéã¿ããåãé€ãã°ãæåããªãã«ã§ãããšããããã§ãã
ç¹°ãè¿ãç¶ããã¢ãã«
ããã¯æ¡æ£ããïŒãã€ã©ã«ã«ãªã£ãïŒå®éšãªã®ã§ã詳ããèŠãŠãããŸãã
ç ç©¶è ã¯å°åã®ãªãŒãã³ãœãŒã¹æšè«ã¢ãã« Qwen3.5-0.8B ãçšããAstra ã«éã¿ç·šéã ãã§æ°åŠæ§èœãé«ããããæ±ããŸãããä»å㯠Astra ã«ã¢ãã«ãèªã¿èŸŒãã Python ããŒãããã¯ãäžããç·Žç¿çšã®æ°åŠåé¡ã§ã¢ãã«ãäžå®åæ°ïŒ10ã25ã50 åã®ãã¹ãå®è¡ãèè ã¯ãã©ã¯ãŒããã¹ãšåŒã¶ïŒãŸã§èµ°ãããããšãèš±å¯ããŸããããã ããããã§ãåŠç¿ã¯äžåçŠæ¢ã§ãã
Astra ã¯ã¢ãã«ããã°ãã°ã«ãŒãã«é¥ãããšãã€ãŸãåãèªãå»¶ã ãšç¹°ãè¿ããŠçãã«å°éããªãããšãããã«èŠæããŸãããæåã® 96 ä»¶ã®ç·Žç¿åçã®ãã¡ 85 ä»¶ãã32,768 ããŒã¯ã³ã®äžéã«éãããŸã§åºåãç¶ããŠããŸããã
ããã¯ããç¥ãããåé¡ã§ãQwen ã®ã¢ãã«ã«ãŒãã§ã泚æåèµ·ãããŠããŸãã2025 幎 12 æã®è«æãWait, Wait, Wait⊠Why Do Reasoning Models Loop?ãã¯ãå系統ã®äžã§å°åã¢ãã«ã»ã©ã«ãŒãããããã倧åã¢ãã«ããèžçãããå°åã¢ãã«ã¯å ã®ã¢ãã«ããããã«ã«ãŒããããããšå ±åããŠããŸãã
Astra ã¯ã«ãŒãã®åå ãã¢ãã«å éšã®ã²ãšã€ã®é ãããã¿ãŒã³ãããªãã¡ç ç©¶è ãæ¹åïŒdirectionïŒãšåŒã¶ãã®ã«çªãæ¢ããŸãããã€ã¡ãŒãžããããããã«èšãã°ãã¢ãã«å éšã®é ããã¹ã©ã€ããŒã§ããä»åã®ã¹ã©ã€ããŒã¯èªå·±å埩ãžãšæŒããããã®ã§ãã¬ã€ã€ãŒ 15 ãš 19 ã«ååšããŸããã
ããã§ Astra ã¯ããã®ã¹ã©ã€ããŒã 131,072 åã®éã¿ïŒ2 ã€ã®å±€ããããã§ 64 åã®åºåïŒããåãé€ããŸãããããã¯ã¢ãã«å šäœã®ããã0.016%ã«çžåœããŸãã
èåŸã«ããæ°åŠ
ãã® Qwen ã®ä¿®æ£ã¯ãæ¢åã®ãã¯ããã¯abliterationïŒå逿³ïŒã«éåžžã«ãã䌌ãŠããŸãã2024 幎ã«ã¯ãã¢ãã«ã®æåŠçãåäžã®æ¹åã§å¶åŸ¡ãããŠããããšã瀺ããããã®æ¹åãéã¿ããåé€ããããšã§æåŠãæ¢ãããããšãããŸããã
èåŸã«ããæ°åŠã¯é©ãã»ã©ç°¡æœã§ã
rãåãé€ãããåäœé·ã®æ¹åãšãããšãããã 1 è¡ã§ããW_new = W - r (ráµ W)çè§£ãå©ãããããå°ããªäŸãæããŸããããéã¿ã
(3, 2)ã® 2 æåãããªããšããŸããæåã®æ°ã¯ã¢ãã«ãã«ãŒãã«åããŠæŒãæåã2 çªç®ã¯ãã以å€ã®æ©èœã ãšããŸããã«ãŒãæ¹åã¯
r = (1, 0)ãã€ãŸãã第 1 æåãã§ããäžåŒã¯ã«ãŒãæåïŒ3ïŒãåãé€ãã(0, 2)ãæ®ããŸããã«ãŒãèŠçŽ ã¯æ¶ãããã以å€ã¯ãŸã£ããå€ãããŸãããéèŠãªã®ã¯ãabliteration ãæ°çŸä»¶ã®ããã³ããäŸãå¿ èŠãšããã®ã«å¯ŸããAstra ã¯ã¢ãã«ãç·Žç¿åé¡ã§èµ°ãããéã¿ãæšè«ããããšã§ãã®ã«ãŒãæ¹åãèªåã§èŠã€ããç¹ã§ãã詳现ãªçµæã¯ä»¥äžã®éãã§ãïŒ8 å®éšã暪䞊ã³ã§èŠããå Žåã¯ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°æŠèŠããåç §ãã ããïŒã
å³ 5: Astra ç·šéååŸã® Qwen3.5-0.8Bãç»åã¯çè äœæã
ããã§ 2 ç¹ã«æ³šç®ããŠãã ããã第äžã«ãã«ãŒãã¯åãªãæ°åŠã®åé¡ã§ã¯ãªããããä¿®æ£ã¯ã¢ãã«ã®æç€ºè¿œåŸãå€§å¹ ã«æ¹åããŸããïŒ50 åçã§ IFEval 㯠21.0% ãã 62.3%ïŒã第äºã«ã10 åã 25 åã®ãã¹ãã§ã¯ã«ãŒããã»ãŒæ®ãã倧ããªè·³ãäžãã㯠50 åã§ã®ã¿èµ·ããŸããã
ä¿®æ£ãéçšããªããšããïŒä¿®æ£ã¯æ¹åãšã¯éã
ããã§åŒ·èª¿ããŠããããã®ã¯ãã«ãŒããæ¢ããããšã¯ã¢ãã«ãæ°åŠãåŸæã«ããããšãšã¯å¥ã ãšããç¹ã§ããããã¯ãã¢ãã«ããã§ã«è§£ããã¯ãã®åé¡ã«ã€ããŠãæåŸãŸã§åçã§ããããã«ããã«éããŸããã
ããã確ããããããç ç©¶è ã¯Gemma 3 270M ã§ãåãå®éšãè¡ããŸãããããã¯ã»ãšãã©ã«ãŒãããªãå°åã¢ãã«ã§ãïŒããŒã¯ã³äžéå°é㯠8.5%ãQwen 㯠93%ïŒããã®å ŽåãAstra ã¯ã»ãšãã©æ¹åã§ããŸããã§ãããMATH-500 ã§ã¯ãã©ã®ç·šéã§ãã¹ã³ã¢ã¯ 1 ãã€ã³ãçšåºŠããåãããAstra èªèº«ãææãªæ¹åãšã¯èªããŸããã§ããã
Astra ã Gemma ã®èª€ããåæãããšã誀ã£ãç®è¡ãä»£æ°æé ãªã©éåžžã®ãã¹ãèŠã€ãããŸããããªãã«ãã¹ãåäžã®å£ããéšåã¯ååšããããã¯ãã㯠4 ãæŽ»ãããäœå°ã¯ãããŸããã§ããã
èŠããã«ãéã¿ç·šéã¯ã¹ãã«ã®çºçŸã劚ããèŠå ãåãé€ãããšã¯ã§ããŠããã¢ãã«ã«ååšããªãã¹ãã«ãä»äžããããšã¯ã§ããŸããã
ã¹ãã« 3: LLM ã¯ã©ããã£ãŠéã¿ãæžããïŒãã¯ãã㯠5ïŒ
ããããããå人çã«æãé©ããéšåã§ããAstra ã¯ãã¹ãŠã®éã¿ããŒãã®ãããã¯ãŒã¯ããåºçºããæ°å€ãèªåã§éžã°ãªããã°ãªããŸããã
ãã¯ãã㯠5: èšèšïŒãã©ã³ïŒããŠããæ°å€ã«èœãšã
æ¹æ³ã¯åçŽã§ããAstra ã¯ãŸãããã°ã©ã ãèšèšããåå±€ãäœãããã¹ãããæ±ºãããã®åŸã«åå±€ããã®æ©èœãæããããéã¿ãéžã³ãŸãã
ç ç©¶è ãã¡ã¯ç°¡ç¥åãã Atari ã²ãŒã éã®MinAtarãçšããBreakoutãSeaquestãAsterix ãéžã³ãŸãããåã²ãŒã ã«ã€ã㊠Astra ã«æž¡ããã®ã¯ä»¥äžã§ãã
åŠç¿ããŒã¿ã¯ãªããããŒã«ããªãããããã¯ãŒã¯ãã²ãŒã ã§ãã¹ãããããšããäžå¯ãAstra 㯠1 åã®è¿ä¿¡ã®äžã§å šéã¿ãæã¡èŸŒã¿ãç ç©¶è ããã®çµæããã¹ãããŸããã
ããšãã° Breakout ã§ã¯ããã©ã³ã¯æ¬¡ã®ãããªãã®ã§ããã
ã芧ã®éããAstra ã®ãããã¯ãŒã¯ã¯3 ã²ãŒã ãã¹ãŠã§1 åã¹ãããåŠç¿ã®ãããã¯ãŒã¯ãäžåããŸããã5 åã¹ãããåŠç¿ã®ãããã¯ãŒã¯ãšæ¯ã¹ãŠããBreakout ã§ã¯çŽ 20 åã®ã¹ã³ã¢ãAsterix ã§ã¯èèãSeaquest ã§ã¯çŽ 60% ã®ã¹ã³ã¢ã«éããŸããã
å²ç¢ãã¬ã€ã€ãŒã®æ§ç¯
äžã®ã²ãŒã çšãããã¯ãŒã¯ã¯ãAstra ã 1 åã®è¿ä¿¡ã§æã¡èŸŒããçšåºŠã®å°ããã§ãããå²ç¢ã§ã¯ã¯ããã«é£åºŠãäžããçŽ 150 äžåã®éã¿ãæã€ãããã¯ãŒã¯ãçšæããŸããããã㯠Astra ã®ããã³ããã«åãŸããããªã倧ããã§ãã
ããã§ä»£ããã«ãAstra ã« Python ã¯ãŒã¯ã¹ããŒã¹ãäžããéã¿ïŒåæç¶æ ã¯ãã¹ãŠãŒãïŒãé²èЧã»ä¿åã§ããããã«ããŸãããããããã¯ãŒã¯ãå®è¡ããããšã¯äžå¯ãšããŸããããããã¯ãŒã¯ããã¹ããããå Žåã¯ãäžéã®ãããã¹ãå®è¡åæ°ïŒ0ã50ïŒãæ¶è²»ããªããã°ãªããŸãããåãã¹ãå®è¡ã§ã¯ 9Ã9 ç€ã§ 128 å±ããã¬ã€ããŸãã
Astra ã¯ãã®ãããã¯ãŒã¯ããå²ç¢ã®ããã°ã©ã ã®ããã«æ§ç¯ããŸããã
ããã¯ã³ã³ãã€ã©ã«ãšãŠã䌌ãŠãããšæãããããããŸãããããã®éãã§ããDeepMind ã®Tracr ãè¡ããããã°ã©ã ãæ£ç¢ºãªãã©ã³ã¹ãã©ãŒããŒã®éã¿ã«èœãšã蟌ãããšãšéåžžã«è¿ãã®ã§ããéãã¯ãTracr ã¯å°çšèšèªã§æžãããããã°ã©ã ãå¿ èŠãšããäžæ¹ãAstra ã¯èªç¶èšèªã®ç®æšããããã°ã©ã èªäœãèæ¡ããç¹ã§ãã
ãã¹ãã倧ããªéããçãçç±
ãããŸã§èªãã§ãå®éšããšã«ã«ãŒã«ãéã£ãŠããããšã«ãæ°ã¥ãã§ããããã²ãŒã èå¥ã»å³åœ¢ã»è»ã»MinAtar ã®å®éšã§ã¯ Astra ã¯ããŒã«ãäžåãªãã§ããããèšèªã»å²ç¢ã»è¿åçã¢ãã«ã»Qwen ã®å®éšã§ã¯ Python ã䜿ããæã«ãããã¯ãŒã¯ã®ãã¹ããå¯èœã§ããã
ç§ã®èŠç«ãŠã§ã¯ãããã¯æçš¿å šäœã§èŠéããããã¡ãªç¹ã§ããããã¹ãåæ°ã倧ãã圱é¿ããŠããããã§ãã
ãããã£ãŠãããŒã«ãªãã®çµæã¯æãé©ãã¹ããã®ã§ãããå®éšå士ãå ¬å¹³ã«æšªæ¯èŒã¯ã§ããªããšããããšã§ããããŸãã
GPT-6 Astra ã¯å®éã«ã©ããã£ãŠããã®ãïŒ
å šãã¯ããã¯ã«å ±éãããã¿ãŒã³ããããŸããAstra ã¯äººéã®ãšã³ãžãã¢ã®ããã«æ¯ãèããŸãã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã° vs. ãã¡ã€ã³ãã¥ãŒãã³ã°ïŒã¢ãã«ç·šé
ãã¡ã€ã³ãã¥ãŒãã³ã°ãä»ã®ç·šéææ³ãšäœãéãã®ãããšæããããããããŸããã以äžã®è¡šã¯æãè¿ãææ³ãæ¯èŒããããããã 5 ã€ã®ãã¯ããã¯ã®ã©ãã«è¿ããã瀺ããŠããŸãã
ããã€ãè£è¶³ããŸãã
倧ããªéãã¯ãWeight Oracles ããã®ä»äºå°çšã«èšç·Žãããã®ã«å¯ŸãããŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯æ±çšã® LLM ã䜿ãç¹ã§ãã
ççŽã«èšãã°ãæ°ããã®ã¯ãéã¿ãç·šéã§ãããããšèªäœã§ã¯ãããŸãããããã¯äœå¹Žãåããç¥ãããŠããŸãããæ°ããã®ã¯ãã²ãšã€ã®æ±çšã¢ãã«ãæšè«ã ãã§ããã倿§ãªäœæ¥ããã£ãŠã®ããããšã§ãã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã®éç
çµæã¯åºæ¿çã§ãããæ éã«èªãããšãéèŠã ãšèããŸããèè ãèªèº«ããããåæã®çµæããšè¿°ã¹ãŠããŸããäž»ãªæžå¿µç¹ã¯æ¬¡ã®éãã§ãã
1. æ¹åã§ã¯ãªãä¿®æ£ã§ãã
Gemma ã®å¯Ÿç §å®éšã瀺ãéããQwen ã®å€§å¹ åäžã®å€§åã¯ã«ãŒã忢ã«ãããã®ã§ããèè ãã®ãã°ãåæãã XenoSpectrum ã®å ±åã§ã¯ãç·šéåŸã«æ°ãã«è§£ãã 579 ä»¶ã®ç·Žç¿åé¡ã®ãã¡ã576 ä»¶ã¯ä»¥åã¯ããŒã¯ã³äžéã§æã¡åãããŠããŸããã
2. æ€èšŒããããã³ãã£ã¢ LLM 㯠1 ã€ã ã
æçš¿ã®å šå®éšã§ GPT-6 Astra ã䜿ãããŸãããSimon Willison ã¯ããã®åŸãªãŒãã³ãŠã§ã€ãã® Qwen 3.8 27B ãçšããããŒã PC äžã§ã²ãŒã èå¥ã¿ã¹ã¯ãåçŸããŸãããã1 ã€ã®ã²ãŒã ã¯å€ããŸãããä»ã®ããã³ãã£ã¢ã¢ãã«ãä¿®çãæžã蟌ã¿å®éšã§ Astra ã«äžŠã¹ããã¯ãèè ããèšç»ãããã³ãããŒã¯ãçããã§ãããã
3. æ»èªãåããŠããªã
ããã¯è«æã§ã¯ãªãããã°èšäºã§ããèè ãã¯å®å šãªãã°ãå ¬éããŠãããHugging Face ã® Ahsen Khaliq ã¯ç·šéæžã¿ Qwen ã¢ãã«ãåæ§ç¯ããŠè©äŸ¡ãåå®è¡ããŸãããèè ã«ããã° MATH-500 㯠51.0%ãããŒã¯ã³äžéå°é㯠31.6% ã§ãããããã¯å ¬éãã§ãã¯ãã€ã³ãã®æçšãªæ€èšŒã§ãããAstra èªèº«ãç·šéãè¡ãéçšã®ç¬ç«åçŸãšã¯ç°ãªããŸãã
4. å ±åçµæãæå©å¯ãã®å¯èœæ§
æ¹è©å®¶ã¯ãåæã®ããã³ãã詊è¡ãå ±åçµæã«å«ãŸããŠããªãç¹ããAstra ã®æšè«åŽåãå®éšéã§ã°ãã€ããŠããç¹ãææããŠããŸããããã³ããã®èšãåããå°ãå€ãããšãã®æåºŠè©ŠéšãèŠåœãããŸããã§ããã
5. ã³ã¹ããäžæ
æè¯ã® Qwen å®è¡ã¯ 158 ã¿ãŒã³ã20.3 æéãèŠãããã®ãã¡ 4.7 æéã¯ãµãŒãã¹åæ¢ã«ãããã¹ã§ãããéåžžã®ãã¡ã€ã³ãã¥ãŒãã³ã°ãšã®æ¯èŒããªããããæéãã³ã¹ãã®ç¯çŽã«ãªããã¯çŸæç¹ã§ã¯äžæã§ãã
6. æ¯èŒãããå¯å®¹
ã²ãŒã çµæã®æ¯èŒå¯Ÿè±¡ã¯åŠç¿æžã¿ãããã¯ãŒã¯ã§ãããåçŽãªææžãã³ãŒããšã®æ¯èŒã¯ãããŸããããããã¯åŽ©ãã§ãããŒã«ã远ãã ããã®ãã¬ã€ã€ãŒãæžãã®ã¯é£ãããªããããåŠç¿æžã¿ãããã¯ãŒã¯ã«åã£ããšãã£ãŠãæåã®å°è±¡ã»ã©é©ãã§ã¯ãªããããããŸãããèè ãã¯ãåããŒã¹ã©ã€ã³ãããããã®å®éšã®çã䞻匵ïŒçµéšãŒãã§ã©ããŸã§è¡ããããªã©ïŒã ããæ€èšŒããŠãããšå¿çããŠããŸãã
èè ãã¯ãè¿åçã¢ãã«ã Gemma ã®çµæãªã©ããŸããããªãã£ãç¹ãççŽã«è¿°ã¹ãŠããããã®ç¹ã¯è©äŸ¡ããŸãããã ãèŠåºãã«ãªãæ°å€ã¯ãäžè¬åã®èšŒæãšããããææãªåæãµã€ã³ãšããŠåãæ¢ããã¹ãã§ãããã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã®å¿çš
ãããã®å¶çŽããã£ãŠããä»åŸããã«é²å±ããã°ã©ããžåããããæ³åãããšããããããŸããèè ããæããå¯èœãªå¿çšäŸã¯æ¬¡ã®éãã§ãã
æåŸã®ç¹ããããã®ç ç©¶ã«èª°ããååãã§ã¯ãªãçç±ã§ããPauseAI ã® CEOãMaxime Fournes ã¯ãã®ç ç©¶æ¹åãç¡è¬ãšåŒã³ãŸãããå®éã®é²å±ãå±éºãª AI éçºãå éãåŸãããã§ãããã ãããã®æçš¿ã¯ãããªãèªå·±æ¹åã瀺ããŠã¯ããªãããšã¯ææããŠãããŸãã
ããååããªå®å šé¢ããããŸããLLM ãè¿åæ§ãããã¯ãã¢ã®ãããªäºè±¡ãéã¿ã®èªã¿åãã ãã§ç¢ºå®ã«çºèŠã§ããããã«ãªãã°ãå ¬éåã®ã¢ãã«æ€æ»ã«æçšãªææ®µãšãªãããŸãã
ä»åŸã¯ïŒ
èè ãã¯çŸåšãåçš® LLM ã®å®åãæž¬ããã³ãããŒã¯ãšããããæ¯æŽããããŒãã¹ãæ§ç¯äžã ãšè¿°ã¹ãŠããŸããä»åŸåãçµãåããšããŠã¯ã次ã®ãããªãã®ããããŸãã
å ¬é以åŸãé²å±ã¯éãããã§ãã11 æ 12 æ¥ã®æŽæ°ã§ã¯ãããããæ¹çãéååè·¯ãžã®æ¡åŒµãå«ã 100 ä»¶è¶ ã®è¿œè©Šã»æ¡åŒµãååšããIWIBench ãšãããã³ãããŒã¯ãç»å Žãããšå ±åãããŠããŸãã
ãèªèº«ã§ç¢ºããããå Žåã¯ãGitHubã§å šå®éšã®å®å šãªãã°ïŒAstra ã«éã£ãå šããã³ãããšæšè«ã®èŠçŽãå«ãïŒãé²èЧã§ããŸããä¿®æ£æžã¿ Qwen ã¢ãã«ã Hugging Face äžã«ããããã©ãŠã¶ã§è©Šããã©ã€ããã¢ããå èšäºãããªã³ã¯ãããŠããŸãã
ãŸãšã
ãŠã§ã€ãã¹ããŒã¹ã»ããã°ã©ãã³ã°ã¯ãåæçãªããè£ä»ããåºãŠããæ°ããã¢ã€ãã¢ã§ããæ±çš LLM ããåŠç¿ãªãã§å°ããªãããã¯ãŒã¯ã®æ©èœãèªã¿åããç«ã¡åŸçããè»ãã«ãŒãããèšèªã¢ãã«ãçŽããããã«ã¯ãŒãããã²ãŒã çšãããã¯ãŒã¯ãæ§ç¯ããŸããã
æãé©ããã®ã¯ããã®ããæ¹ã§ããè¬ãããããªãã¯ã§ã¯ãªãããããã¯ãŒã¯ã®ç«¯ãèªã¿ãäžã®ãµãŒããããæ¢ããã·ã§ãŒãã«ãããè¶³ããæ¹åãåããæ°å€åããåã«ããã°ã©ã ãèšèšããââãŸãã§ãšã³ãžãã¢ã®ããã§ããã
ãã ããã㯠1 ã€ã®ã¢ãã«ã«åºã¥ãåæå ±åã§ãããæå€§ã®ææã¯ã¢ãã«ãè³¢ããããšãããããã°ä¿®æ£ã«è¿ããã®ã§ããç¹ã¯æ¹ããŠåŒ·èª¿ããããšæããŸããä»åŸãé²å±ããã°ãæãæçšãªã®ã¯åŠç¿ã®ä»£æ¿ã§ã¯ãªããã¢ãã«ã®ç£æ»ããããã°ã«ãªãã§ãããã
ãã®çš®ã®ç ç©¶ã远ãçŽ å°ãæŽãããå Žåã¯ãLarge Language Models (LLMs) Concepts ã§ã¢ãã«ã®å åŽã®ä»çµã¿ãåŠã¹ãŸãããšãŒãžã§ã³ãåŽã«é¢å¿ããããªããIntroduction to AI Agents ãè¯ã次ã®äžæ©ã§ãã