AlphaGo Zero

2017年10月19日凌晨，在國際學術期刊《自然》（Nature）上發表的一篇研究論文中，谷歌下屬公司Deepmind報告新版程序AlphaGo Zero：從空白狀態學起，在無任何人類輸入的條件下，它能夠迅速自學圍棋，並以100:0的戰績擊敗“前輩”。 ^[1]  它經過3天的訓練便以100：0的戰績擊敗了AlphaGo Lee，經過40天的訓練便擊敗了AlphaGo Master。 ^[1] 

“拋棄人類經驗”和“自我訓練”並非AlphaGo Zero最大的亮點，其關鍵在於採用了新的reinforcement learning（強化學習的算法），並給該算法帶了新的發展。 ^[1] 

AlphaGo Zero僅擁有4個TPU，零人類經驗，其自我訓練的時間僅為3天，自我對弈的棋局數量為490萬盤。但它以100:0的戰績擊敗前輩。 ^[1] 

A long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains. Recently, AlphaGo became the first program to defeat a world champion in the game of Go. The tree search in AlphaGo evaluated positions and selected moves using deep neural networks. These neural networks were trained by supervised learning from human expert moves, and by reinforcement learning from self-play. Here we introduce an algorithm based solely on reinforcement learning, without human data, guidance or domain knowledge beyond game rules. AlphaGo becomes its own teacher: a neural network is trained to predict AlphaGo’s own move selections and also the winner of AlphaGo’s games. This neural network improves the strength of the tree search, resulting in higher quality move selection and stronger self-play in the next iteration. Starting tabula rasa, our new program AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo. ^[3] 

人工智能的一個長期目標是一種算法，它可以在具有挑戰性的領域中學習超人的熟練程度。最近，AlphaGo成為第一個在圍棋比賽中擊敗世界冠軍的程序。AlphaGo的樹搜索使用深度神經網絡評估位置和選定移動。這些神經網絡是通過人類專家動作的監督學習和自我對弈的強化學習來訓練的。在這裏，我們介紹了一種僅基於強化學習的算法，沒有超出遊戲規則的人類數據、指導或領域知識。AlphaGo成為自己的老師：訓練神經網絡來預測AlphaGo自己的走法選擇以及AlphaGo遊戲的獲勝者。該神經網絡提高了樹搜索的強度，從而在下一次迭代中產生更高質量的移動選擇和更強的自我對弈。從白紙開始，我們的新程序AlphaGo Zero取得了超人的表現，以100-0戰勝了之前發佈的擊敗冠軍的AlphaGo。 ^[4]