An Efficient Deep Q-learning Strategy for Sequential Decision-making in Game-playing

Oscar Chang, Manuel Eugenio Morocho-Cayamcela, Israel Pineda, Kevin Cardenas

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

This paper presents a deep reinforcement learning model that efficiently learns a sequential decision-making policy to play tic-tac-toe intelligently directly from a high-dimensional video. To produce a stable, sparse neural representation of the states of the tic-tac-toe board, a convolutional pre-trained neural network has been used, followed by a fully-connected sigmoidal network. The assemble behaves as a Q-matrix and produces the ultimate state-decision pairs that control a robotic arm placing physical tokens on the board. The hyperparameters in the whole network are tuned to produce a stable trainable array of elements. An internal clock composed of internal neurons is integrated to give the agent a sense of sequential timing. To solve the max(⊙) function, a novel algorithm is introduced to search for the Q-network values. The algorithm uses a dedicated, sigmoidal net initialized with random parameters. Under backpropagation it iteratively moves to a stable plateau that mimics the all-zeros condition of an initial Q-matrix. Next, the agent uses Bellman's reinforcement principles to learn an optimal policy with a noticeable look-ahead capability. Computer simulations driving a physical robot proved the convergence and effectiveness of the proposed methodology and demonstrated a marked ability in sequential decision-making, taking raw video frames as input.

Original languageEnglish
Title of host publicationProceedings - 3rd International Conference on Information Systems and Software Technologies, ICI2ST 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages172-177
Number of pages6
ISBN (Electronic)9781665455176
DOIs
StatePublished - 2022
Event3rd International Conference on Information Systems and Software Technologies, ICI2ST 2022 - Quito, Ecuador
Duration: 8 Nov 202210 Nov 2022

Publication series

NameProceedings - 3rd International Conference on Information Systems and Software Technologies, ICI2ST 2022

Conference

Conference3rd International Conference on Information Systems and Software Technologies, ICI2ST 2022
Country/TerritoryEcuador
CityQuito
Period8/11/2210/11/22

Keywords

  • Artificial intelligence
  • deep Q-learning
  • game-playing
  • reinforcement learning
  • sequential decision-making

Fingerprint

Dive into the research topics of 'An Efficient Deep Q-learning Strategy for Sequential Decision-making in Game-playing'. Together they form a unique fingerprint.

Cite this