TITLE:  Dynamic and Neuro-Dynamic Programming: An Overview and Recent Work

  SPEAKER:   Dimitri P. Bertsekas

McAfee Professor of Engineering
Lab. for Information and Decision  Systems
Massachusetts Institute of Technology
Cambridge, MA 02139

Dynamic programming is a broadly applicable methodology for sequential decision making, but suffers from exponential growth of computational requirements as the problem size increases. This has led to extensive work on approximations over the last twenty years. One key idea is to use an (approximate) scoring function to select decisions in complex dynamic systems, arising in a broad variety of applications from engineering design, operations research, resource allocation, finance, etc. This is much like what is done in computer chess and computer backgammon, where positions are evaluated by means of a scoring function and the move that leads to the position with the best score is chosen. Neuro-dynamic programming/reinforcement learning provides a class of systematic methods for computing appropriate scoring functions using neural network-like approximation schemes and simulation/evaluation of the system's performance. Another important idea is to use heuristics to compute on-line the values of an approximate scoring function, and is well-suited for large discrete optimization problems. The talk will overview these methodologies and discuss recent work.

Dimitri P. Bertsekas received a combined B.S.E.E. and B.S.M.E. from the National Technical University of Athens, Greece, an M.S.E.E. from George Washington University, and a Ph.D. in system science from the Massachusetts Institute of Technology. He has held faculty positions with the Engineering-Economic Systems Dept., Stanford University (1971-1974) and the Electrical Engineering Dept. of the University of Illinois, Urbana (1974-1979). Since 1979 he has been teaching at the Electrical Engineering and Computer Science Department of the Massachusetts Institute of Technology (M.I.T.), where he is currently McAfee Professor of Engineering. He consults regularly with private industry and has held editorial positions in several journals. His research at M.I.T. spans several fields, including optimization, control, large-scale computation, and data communication networks, and is closely tied to his teaching and book authoring activities. He has written numerous research papers, and thirteen books, several of which are used as textbooks in MIT classes.

Dr. Bertsekas was awarded the INFORMS 1997 Prize for Research Excellence in the Interface Between Operations Research and Computer Science for his book "Neuro-Dynamic Programming" (co-authored with John Tsitsiklis), the 2000 Greek National Award for Operations Research, and the 2001 ACC John R. Ragazzini Education Award. In 2001, he was elected to the United States National Academy of Engineering.

Dr. Bertsekas' recent books are "Introduction to Probability" (2002), "Convex Analysis and Optimization" (2003), "Dynamic Programming and Optimal Control: 3rd Edition" (2007), all published by Athena Scientific. He is writing a new book on Convex Optimization Theory (to appear in 2008).

TITLE:   The Challenges of Cognitive Interaction Technology

SPEAKER:      Helge Ritter

Faculty of Technology
Bielefeld University

While the rapid advances in technology are bringing the complexity of technical systems closer to the level of biology, the interaction between humans and such systems raises new challenges, one of the foremost being how to facilitate the guidance and use of such systems and endow it with the ease we are accustomed from natural cooperation and communication between humans. We argue that the realization of that goal will require a basic understanding of how to synthesize the quality of cognitive interaction from more realizable constituents that cover substantial partial functions such as intelligent motion, attention, situated communication and memory with learning. We point out some exemplary research questions and report on ongoing research that led to the Bielefeld-based research initiative "CITEC - Cognitive Interaction Technology" launched recently in the context of the German Excellence Initiative, along with the closely associated "Cognition and Robotics Lab" (CoR-Lab), both bringing together an interdisciplinary consortium of computer scientists, biologists, linguists and psychologists aiming towards elucidating principles of cognitive interaction and their replication in technical systems.

Biography

Helge Ritter studied physics and mathematics at the Universities of Bayreuth, Heidelberg and Munich. After a Ph.D. in physics at Technical University of Munich in 1988 he visited the Laboratory of Computer Science at Helsinki University of Technology and the Beckman Institute for Advanced Science and Technology at the University of Illinois at Urbana-Champaign. Since 1990 he is head of the Neuroinformatics Group at the Faculty of Technology, Bielefeld University. His main interests are principles of neural computation and their application to build intelligent systems. In 1999, Helge Ritter was awarded the SEL Alcatel Research Prize and in 2001 the Leibniz Prize of the German Research Foundation DFG. He is co-founder and Director of the Bielefeld Cognitive Robotics Laboratory (CoR-Lab) and coordinator of the Bielefeld Excellence Cluster "Cognitive Interaction Technology" (CITEC).

TITLE:  Approximate/Adaptive Dynamic Programming and Their Applications

  SPEAKER:   Jennie Si

Professor of Electrical Engineering
Arizona State University
Tempe, AZ 85287

Dynamic programming (DP) is an approach to computing the optimal control policy over time under nonlinearity and uncertainty by employing the principle of optimality introduced by Richard Bellman. Instead of enumerating all possible control sequences, dynamic programming only searches admissible state and/or action values that satisfy the principle of optimality. Therefore, the computation complexity can be much improved over the direct enumeration method. However, the computational efforts and the data storage requirement increase exponentially with the dimensionality of the system, which are reflected in the three curses: the state space, the observation space, and the action space. Thus, the traditional DP approach was limited to solving small size problems. This talk aims at exploring a class of approximate/adaptive dynamic programming algorithms that are especially useful in continuous state and continuous control problems. The talk will review these algorithms, their implementations and properties, as well as how to apply them to large, realistic engineering problems. The talk will also examine the feasibility of modeling the neural basis of decision making and control strategy development in behaving animals using these approximate/adaptive dynamic programming paradigms.

Biography

Jennie Si received her B.S. and M.S. degrees from Tsinghua University, Beijing, China, and her Ph.D. from the University of Notre Dame. She has been on the faculty in the Department of Electrical Engineering at Arizona State University since 1991. Her research focuses on dynamic optimization using learning and neural network approximation approaches. This entails fundamental understanding of learning and adaptive systems and development of learning algorithms. In addition, she is interested in applications of her systems knowledge in large physical systems such as semiconductor processes and biological neural systems. Recently she set up a neurophysiology lab using chronic multi-channel recording to study the neural mechanism of decision and control in rat’s motor cortical areas. Jennie Si received the NSF/White House Presidential Faculty Fellow Award in 1995. She also received a Motorola Engineering Excellence Award in the same year. She is a Fellow of the IEEE. She is past Associate Editor of the IEEE Trans. on Automatic Control; past Associate Editor of the IEEE Trans. on Semiconductor Manufacturing, and current Associate Editor of the IEEE Trans. on Neural Networks.

TITLE:  Neural Computing in Web Search

  SPEAKER:   Hang Li

Microsoft Research Asia

Search is becoming the major means for people to access the Internet. According to a survey, about 55% of internet users use search engines every day. Web search engines are usually built with technologies from two areas, large-scale distributed computing and statistical learning. Statistical learning is useful because there are many uncertainties in crawling, indexing, ranking, and serving of web search and the solutions to them have to be data-driven. In this talk, I will explain how statistical learning technologies are being used in web search. Specifically, I will introduce some of the neural computing  technologies for web search, developed at Microsoft.

Biography

Hang Li is senior researcher and research manager at Microsoft Research Asia. He is also adjunct professor at Peking University, Nanjing University, Xian Jiaotong University, and Nankai University. His research areas include natural language processing, information retrieval, statistical machine learning, and data mining. He graduated from Kyoto University and earned his PhD from the University of Tokyo. Hang has over 60 publications in international journals and conferences. He is associate editor of ACM Transaction on Asian Language Information Processing and is in editorial board of Journal for Computer and Science Technology, Journal of Chinese Information Processing, etc. His recent academic activities include program committee co-chair of AIRS’08, poster and demo committee co-chair of SIGIR’08, program committee area chair of PAKDD’08, etc. Hang has been working on development of several products. These include NEC TopicScope, Microsoft SQL Server 2005, Microsoft Office 2007.