Probabilistic Robot Navigation in Partially Observable Environments

Transcription

1 Pobabiistic Robot Navigation in Patiay Obsevabe Envionments Reid Simmons and Sven Koenig Canegie Meon Univesity Schoo o Compute Science Pittsbugh, PA eids@cs.cmu.edu, skoenig@cs.cmu.edu Abstact Autonomous mobie obots need vey eiabe navigation capabiities in ode to opeate unattended o ong peiods o time. This pape epots on ist esuts o a eseach pogam that uses patiay obsevabe Makov modes to obusty tack a obot s ocation in oice envionments and to diect its goa-oiented actions. The appoach expicity maintains a pobabiity distibution ove the possibe ocations o the obot, taking into account vaious souces o uncetainty, incuding appoximate knowedge o the envionment, and actuato and senso uncetainty. A nove eatue o ou appoach is its integation o topoogica map inomation with appoximate metic inomation. We demonstate the obustness o this appoach in contoing an actua indoo mobie obot navigating coidos. 1 Intoduction We ae inteested in the task o ong-tem autonomous navigation in an oice envionment (with coidos, oyes, and ooms). Whie the state o the at in autonomous oice navigation is aiy advanced, it is not geneay good enough to pemit obots to tavese coidos o ong peiods o time without getting ost. Evidence o this can be seen in ecent AAAI-sponsoed obot competitions [Konoige, 1994; Simmons, 1995], whee the obots oten got conused as to whee they wee, and had diicuty eocaizing once that occued. We contend that navigation can be made moe eiabe by having the obot expicity epesent spatia and senso uncetainty. To this end, we have deveoped a navigation technique that uses Makov modes to obusty tack the obot s position and diect its couse. A patiay obsevabe Makov de- This eseach was suppoted in pat by NASA unde contact NAGW-1175 and by the Wight Laboatoy and ARPA unde gant numbe F The views and concusions contained in this document ae those o the authos and shoud not be intepeted as epesenting the oicia poicies, eithe expessed o impied, o NASA, the Wight Laboatoy, o the United States govenment. cision pocess (POMDP) mode is constucted om topoogica inomation about the connectivity o the envionment, appoximate distance inomation, pus senso and actuato chaacteistics. The Makov mode estimates the position o the obot in the om o pobabiity distibutions. The pobabiities ae updated when the obot epots that it has moved o tuned, and when it obseves eatues such as was and coido junctions. To diect the obot s behavio, a panne associates a diective (e.g., tun o stop) with evey Makov state. Wheneve the pobabiity distibution o the Makov mode is updated, the tota pobabiity mass o each diective is cacuated, and the obot executes the one with the agest pobabiity mass. Ou appoach has sevea eatues that make it we-suited o the oice navigation task. It expicity accounts o uncetainty in actuation, senso data and thei intepetation, and the obot s position. It can utiize a avaiabe senso inomation to tack position, and is paticuay amenabe to adding new souces o senso inomation. It seamessy combines topoogica and metic map inomation, enabing the obot to utiize as much, o as itte, metic inomation as it has avaiabe. It is aso vey eactive once the obot beieves it has stayed om the nomina (optima) path, it wi automaticay execute coective actions. On the othe hand, it is eativey immune to tempoay uncetainty in position. Fo exampe, even i the obot does not know o cetain which o two paae coidos it is tavesing, it does not stop and epan, as ong as the conto diectives associated with both coidos ae the same. In this way, it can continue making pogess towads its desied goa, whie at the same time coecting senso eadings to hep disambiguate its tue ocation. An impotant aspect o this wok is that it must un in ea time on boad an actua obot. Pobems incude not ony how to mode the navigation pobem as a POMDP, but aso how to dea with memoy and time constaints. Whie sti peiminay, ou expeimenta esuts, both in simuation and on the actua obot, ae encouaging. In paticua, they indicate that the appoach poduces vey obust navigation, even when using estimates o the actua senso and action modes. Whie, to date, we have concentated moe on impementation and vaidation aspects o the appoach, ou wok opens up new appication aeas o moe theoetica esuts

2 $ in the aea o panning with Makov modes, incuding some o ou own goup s wok [Chisman, 1992; Goodwin, 1994; Koenig and Simmons, 1994]. 2 Reated Wok Most ecent wok in obotic oice navigation has used a andmak-based appoach that eies on topoogica maps whose nodes coespond to andmaks (ocay distinctive paces), such as coido junctions, and whose edges indicate how the obot shoud navigate between nodes [Kotenkamp and Weymouth, 1994; Kuipes and Byun, 1988]. This appoach is attactive because it does not depend on geometic accuacy and is eactive to sensed eatues o the envionment (the andmaks). It sues, howeve, om pobems o sensos occasionay not detecting andmaks and o senso aiasing (not being abe to distinguish between simia andmaks). On the othe hand, using puey metic maps is vuneabe to inaccuacies in both the map making and dead-eckoning abiities o the obot. Whie some eseaches augment topoogica maps with appoximate metic inomation, such inomation is pimaiy used to esove topoogica ambiguities [Kuipes and Byun, 1988; Mataic, 1991; Simmons, 1994]. In contast, ou Makov mode appoach seamessy integates topoogica, andmak-based, inomation and appoximate metic inomation. Some navigation techniques epesent uncetainty in position using modes that pesume a cetain pobabiity distibution, typicay Gaussian [Kosake and Kak, 1992; Smith and Cheeseman, 1986]. Whie such modes ae eicient to encode and update, they ae not ideay suited o oice navigation. In paticua, due to senso aiasing, one oten wants to encode the beie that the obot might be in one o a numbe o non-contiguous ocations. This cannot be epesented pecisey using Gaussian distibutions, but is quite easy o ou Makov modes. On the othe hand, we need to tesseate space into discete states, athe than epesenting position using ea numbes. Thus, thee is a tadeo between the pecision and expessiveness o the dieent modes. We contend that o oice navigation, howeve, that the added expessiveness o the Makov modes outweighs the decease in pecision om discetization. Like ou own wok, sevea eseaches have investigated Bayesian appoaches o pobabiistic panning and execution monitoing in oice navigation. [Noubakhsh et a., 1995] use a patiay obsevabe Makov mode appoach simia to ous, but do not utiize any metic inomation. The states o the obot ae eithe at a topoogica node, o somewhee in a connecting coido. In contast, ou appoach can use estimates o how a the obot has taveed and senso epots that occu within a coido to uthe constain the obot s ocation. Fo exampe, knowing that two coido junctions ae appoximatey 5 metes apat enabes the obot to estimate when it is in the vicinity o the second junction, even i it misses seeing the junction. Most o the othe Bayesian appoaches ey on metic maps. [Kiman et a., 1991] and [Nichoson and Bady, 1994] use appoaches based on tempoa beie netwoks. With such methods the size o the modes gows ineay with the amount o tempoa ookahead, which imits thei use to athe sma ookaheads. [Dean et a., 1993] use obot navigation as an exampe to descibe a panning and monitoing agoithm that uses a totay obsevabe Makov mode, which assumes that the ocation o the obot is aways known pecisey. [Hu and Bady, 1994] use Bayesian techniques to detect unoeseen obstaces in an othewise competey known envionments. 3 Constucting the Makov Modes Beoe descibing how we constuct Makov modes o an oice envionment, we intoduce some teminoogy. A inite Makov mode consists o a inite set o states, a inite set o actions, a set o actions o each state, that can be executed in that state, and tansition pobabiities o a and (the pobabiity that the new state is i action is executed in state ). We aso deine a set o sensos. The sensos ae chaacteized and by obsevation pobabiities! " o a # %& (the pobabiity that senso epots eatue when the obot is in state ). Note that Makov modes assume that the tansition and obsevation pobabiities ae detemined ony by the cuent state o the obot (the Makov assumption ). In ou case, the Makov mode is patiay obsevabe because the obot may neve know exacty which state it is in. Instead, it maintains a beie o its cuent state in om o a pobabiity distibution ove the states. The pobabiity distibution is updated in two ways: When an action epot is eceived, indicating a move o tun, the new pobabiities become: ')(*,+!-,./( *,7%89;: <8>=@?/*,7BA whee 2 C D4 'E./(. is a nomaization acto to ensue that the pobabiities a sum to one (this is necessay because not a actions ae deined o a states). When a senso epot is eceived om senso, indicating that a eatue has been detected, the pobabiities become: ')(F*G+!-,.FH(. I0J254! " K4 'L.F/(M. The Makov mode is constucted om thee souces o inomation: the topoogy o the envionment (which we pesume can be easiy obtained), genea knowedge about oice envionments (such as that coidos ae staight and pependicua to each othe), and appoximate metic knowedge (obtained eithe om ough measuements o om genea knowedge, such as the act that, in ou buiding, coidos ae two metes wide and a dooways ae between two and ten metes apat). The map inomation is initiay encoded as a gaph o nodes and edges (Figue 1). A node epesents a junction between coidos (and/o dooways o oyes). Nodes ae connected by a pai o diected edges, which ae augmented

3 type: coido distance: 2 m with pob m with pob m with pob type: topo node noth: open west: open south: open east: wa a. Makov mode o topoogica edges (i the edge ength is known exacty) topo node topo node b. The paae chains Makov mode (i the edge ength is not known exacty) coesponds to topo node coesponds to topo node Figue 1: Augmented Topoogica Map Figue 2: Goup o Fou Makov States with appoximate ength inomation in the om o a pobabiity distibution ove possibe engths. Rooms and oyes (not shown) ae aso epesented in the map. The est o this section descibes how the augmented topoogica map is compied into a Makov mode. Modeing Locations Each Makov state encodes both the oientation and ocation o the obot. To insuate the mode om ow-eve conto aspects (such as tuning to avoid obstaces), we encode the commanded heading o the obot athe than its instantaneous oientation. Since ou coidos ae staight and pependicua to each othe, it is suicient to discetize oientation into the ou compass diections: Noth, South, East, West. The spatia ocations o the obot ae aso discetized. Whie moe ine-gained discetizations yied moe pecise modes, they aso esut in moe memoy equiements and moe timeconsuming computations. We use a esoution o one mete, which we have ound to be suicient. Since ou Makov states encode both oientation and ocation, ou states ae needed to uy epesent each spatia ocation. Thee actions ae modeed: tuning ight 90 degees ( ), tuning et 90 degees ( ), and going owad one mete ( ). Right and et tun actions ae deined o evey state (Figue 2). Since they coespond to changes in commanded heading and not to changes in position, we have ound it suicient to mode them deteministicay. Some states aso have owad actions deined o tansitioning om ocation to ocation (note that owad actions ae not deined o states that ace was). Dead-eckoning uncetainty is modeed by a se-tansition, that is, the owad action tansitions with some pobabiity om a state into itse. Modeing Coidos Ou epesentation o topoogica edges is a key to ou appoach. I the edge engths ae known exacty, it is simpe to mode the abiity to tavese a coido with a Makov chain that has owad actions between those states whose oientations ae paae to the coido axis (Figue 3a). The mode c. Come om semantics (i the edge ength is not known exacty) A B Figue 3: Repesentations o Topoogica Edges becomes moe compex when ony appoximate edge engths ae known. Whie one appoach is to epesent a coido edge by a singe Makov state [Noubakhsh et a., 1995], this oses the abiity to utiize dead-eckoned inomation in doing position estimation. Anothe appoach is to mode an edge as a set o paae Makov chains, each coesponding to one o the possibe engths o the edge (Figue 3b). The tansition pobabiities into the ist state o each chain ae the same as the pobabiity distibution ove edge engths associated with the topoogica map (see Figue 1). Each owad tansition ate that is deteministic (moduo dead-eckoning uncetainty note that the identity tansitions ae not shown in these igues). Whie this epesentation best captues the actua stuctue o the envionment, it is eativey ineicient: the numbe o states is quadatic in the maximum ength o the edges. As a compomise between ideity and eiciency, ou cuent impementation modes edges by coapsing the paae chains in a way that we ca the come om semantics (Figue 3c). In this epesentation, the spatia ocation o a Makov state is known eative to the topoogica node om which the obot comes, but its ocation eative to the end o C

4 C B D Senso Data Senso Intepetation Action and Senso Repots Position Estimation Commanded Heading Pobabiity Distibution Robot Contoe Diectives Action Seection Moto Commands A Goas Panne State-Action Poicy Figue 5: Navigation System Achitectue (o caity, ony actions om the highighted nodes ae shown) Figue 4: Repesentation o Coido Junctions the chain is uncetain (e.g., state B is 1 mete om A, but is between 1 and 3 metes om C). Fo each state, the owad tansition pobabiities ae deived om the edge ength pobabiity distibutions. When edge ength uncetainty is age, the come om semantics can save signiicant space ove the paae chains epesentation. Fo exampe, o an edge between 2 and 10 metes ong, the come om semantics needs ony 80 states to encode the edge, compaed to 188 o the paae chains. Each edge in the come om semantics is actuay epesented using two chains, one o each o the coido diections. Thus, i the obot taves some distance and then tuns aound, the mode imits the positiona uncetainty as the obot taves back to the ast topoogica node. This is paticuay useu when the obot eaizes it has missed a junction, and tuns aound to head back. Modeing Junctions and Dooways Unotunatey, we cannot epesent coido junctions simpy with a singe goup o ou Makov states, since the spatia esoution o a Makov state is one mete, but ou coidos ae two metes wide. Whie one appoach woud be to epesent junctions using ou (two by two) goups o ou Makov states each, we achieve neay the same esut with ou goups o two states each, which both saves space and makes the mode simpe (Figue 4). The basic idea is that tuns within a junction ae non-deteministic, with equa pobabiity o tansitioning to one o the two states o the appopiate oientation in the junction. Fo exampe, in enteing the junction o Figue 4 om the South, the obot woud ist encounte state A, then state B i it continues to move owad. I it then tuns ight, it woud be acing East, and woud tansition to eithe states C o D with equa pobabiity. This modes agees with how the obot actuay behaves in junctions. In paticua, it captues the uncetainty that aises due to the act that the obot tuns with a non-zeo tun adius. Dooways can be modeed much moe simpy, since the width o ou doos is appoximatey the esoution o the Makov mode. A singe exact-ength edge (Figue 3a) eads though a doo into a oom. Simiay to [Noubakhsh et a., 1995], dooway edges have an associated pobabiity that the doo is open. Then, the obsevation pobabiities associated with seeing a dooway ae:! " doo 0 4 ; " open-doo 4 ; " cosed-doo Modeing Foyes and Rooms We ae deveoping adequate modes o age open spaces (oyes and ooms). Cuenty, we tesseate a oye into a matix o ocations. Fom each ocation, the owad action has some pobabiity o tansitioning staight ahead, but aso some pobabiity o se-tansitioning and moving to diagonay adjacent states. Whie this mode coesponds we with ou obsevations about how the obot actuay peoms in such spaces, it is deicient in that it equies the exact ength and width o the oye. Athough this mode coud aso be used to epesent ooms, it is pobaby ovey compex o that pupose: we ae cuenty eaning towads epesenting ooms using a singe goup o ou states, each o which has a high pobabiity o se-tansitioning. 4 The Navigation System Achitectue The ovea system achitectue consists o ive main components (Figue 5). The obot contoe peoms oca obstace avoidance whie tying to tave aong a commanded heading. The senso intepetation component convets aw data om the whee encodes and sona into highe-eve action epots (heading changes and distance taveed) and senso epots (eatues detected). Position estimation uses these epots and the Makov mode to maintain a beie about the cuent ocation o the obot. Action seection uses this pobabiity distibution, aong with a goa-diected poicy poduced by the panne, to choose diectives which ae sent to the contoe to change the obot s heading o make it stop. These diectives ae aso ed back to senso intepetation, since intepetation o eatues is oten heading speciic. To date, the wok epoted hee has ocused on position estimation and action seection. The obot contoe and senso intepetation components ae essentiay the same as those used in ou pevious wok in andmak-based navigation [Simmons, 1994], and we have not yet put signiicant eot into the panne.

5 Sma Opening (Doo) Wa Lage Opening (Intesection) Figue 6: Occupancy Gid with Coido Featues Robot Contoe The main task o the obot contoe is to head in a given diection whie avoiding obstaces. To do that, it uses a potentia ied appoach [Akin, 1987], in which obstaces ae epesented as epusive oces and the desied heading is an attactive oce. The obot sums the oce vectos and ocay moves in that diection, moduating its speed i necessay to avoid coisions. The diectives suppied to the contoe ae to make it stop, go, and change heading. Whie the Makov mode epesents tuns and moves as discete actions, in eaity the obot does not stop to tun, but continuay moves owad, even whie tuning. In addition, heading changes ae cumuative, so that two successive ight tun diectives, o instance, esuts in a smooth 180 degee tun. Senso Intepetation The task o the senso intepetation component is to convet the continua motion o the obot into discete action epots, and to poduce senso epots om the aw senso data that indicate the obsevation o high-eve eatues, such as was and coido openings. The senso intepetation component peiodicay eceives epots om the obot s dead-eckoning, which uses intena sensos (whee encodes) to estimate position and oientation. This inomation is combined with the obot s commanded heading to poduce a vitua odomete that keeps tack o the distance taveed aong that heading. This is needed so that the distance the obot taves in avoiding obstaces is not counted in detemining how a it has taveed aong a coido. Ate each mete o cumuative tave, the senso intepetation epots that one owad action has occued. Simiay, the obot contoe epots when its commanded heading has changed, and this is epoted (in units o 90 degee tuns) to the position estimation component. Sona eadings ae bunded into thee vitua sensos that epot obsevations o was and openings o vaious sizes (sma, medium and age) in ont o the obot and to its immediate et and ight. An occupancy gid [Ees, 1989], which pobabiisticay combines sona senso eadings taken ove time as the obot taves, is used to ite noisy senso eadings and poduce a moe goba view o the obot s suoundings (Figue 6). The occupancy gid is pocessed by pojecting a sequence o ays pependicua to the obot s commanded heading (thus, it is independent o the obot s actua oientation), unti they intesect an occupied gid ce. The ays ae then anayzed geometicay. I the end points o the ays can be it to a ine easonaby we (i.e., with a sma chi-squaed statistic), then a wa has been detected with high pobabiity. An opening is indicated by a contiguous sequence o ong ays. Position Estimation The vitua senso and action epots ae used to update the pobabiity distibution ove the Makov states accoding to the update ues shown in Section 3. These ues need the tansition pobabiities o actions and the obsevation pobabiities o vitua sensos ". The tansition pobabiities ae deived om edge ength distibutions in the map pus knowedge o dead-eckoning uncetainty. The obsevation pobabiities must be estimated o eaned. To simpiy the pobem, athe than chaacteizing each individua state, we chaacteize casses o states, such as wa, open (coido junctions), cosed-doo, and open-doo. Then, we ceate a tabe containing eatue/state cass pais that encode the pobabiity that the senso epots a given eatue when the obot is next to that paticua cass o states. Fo exampe, the et vitua senso is patiay chaacteized by: (wa open) = 0.05 (sma opening open) = 0.20 (medium opening open) = 0.40 (age opening open) = 0.30 (nothing open) = 0.05 These pobabiities indicate that junctions ae most commony detected as medium-sized openings, but can oten be seen as age o sma openings (athough they ae hady eve conused o was). The obsevation pobabiities o the eatue nothing, which is used to indicate that a senso has made no detemination, ae chosen so that, i the sensos epots nothing, the ovea pobabiity distibution is unaected. Whie these vaues epesent ou best guesses, we have impemented eaning agoithms to detemine action tansition and obsevation pobabiities moe pecisey. In genea, owad actions tend to incease positiona uncetainty, due to non-deteministic tansitions, whie obsevations tend to decease it. In cetain cases, howeve, the eects o a owad action can damaticay decease uncetainty. This occus when thee is some pobabiity that the obot is in states with no owad actions ( # ). Such states ae pevaent o instance, a states within a coido whose oientation is pependicua to the axis o the coido (see Figue 3). In pactice, this eect can be seen when the obot tuns at an intesection: Beoe the tun, thee is oten some pobabiity that the obot has not yet eached the intesection. Ate the obot has tuned and successuy moved

6 owad a bit, the pobabiity that it is sti in the oigina coido dops to zeo. We beieve this is a majo acto in keeping the positiona uncetainty ow, even when the obot taves ong distances. When incopoating senso epots, cae must be taken to peseve the Makov assumption. Since epots by the same senso at the same ocation ae not independent (since they depend on the same occupancy gid ces), mutipe epots cannot be aggegated. Instead, we etact the od senso epot beoe updating with the new epot, which can be done easiy as ong as no action updates occu between the two epots. Action Seection To conto the obot s goa-diected behavio, ou panne (see with each Makov beow) associates a diective state (note: these shoud not be conused with the set o actions deined o the Makov mode). The ou diectives ae: change heading by 90 degees (tun ight), -90 degees (tun et), 0 degees (go owad), and stop. The action seection component chooses new diectives based on the pobabiity distibution o the Makov mode. A staightowad stategy is to choose the diective associated with the state that has the highest pobabiity [Noubakhsh et a., 1995]. Whie this stategy may be adequate when each topoogica entity is associated with a singe Makov state, it does not wok we in ou modes. Fo exampe, since coido junctions ae modeed using sevea states o each oientation, it is easonabe to conside a o thei ecommendations when deciding which diective to issue. A seection stategy with this popety is the best-action stategy, in which the pobabiity mass o each diective is cacuated and the one with the highest tota pobabiity is chosen: ag 8 6 *F89;:?H*,A A vaiation on this is the best-above-theshod seection stategy, which chooses the best diective ony i its pobabiity mass is above some theshod, othewise the cuent diective emains in eect. We have investigated this stategy because we thought it woud educe the chances o making wong moves due to spuious ase positive senso epots. Expeimenta evidence, howeve, both in simuation and with the ea obot, indicate that the best-action stategy is in act supeio in educing the numbe o eoneous moves. Reinocement eaning eseaches, such as [Chisman, 1992; Tenenbeg et a., 1992], oten use othe voting schemes, such as the oowing: Let C be the shotest distance om state to the goa i the obot executes diective and then behaves optimay. The stategy chooses the diective with the smaest expected goa distance: ag 8 6 *F89 ; C This scheme aows one, o exampe, to choose the second best action i a states agee on the second best action, but disagee on the best action. Whie this scheme is attactive, we did not impement it because it woud equie substantia changes to ou path panne. Panning Whie oppotunities abound o appying pobabiistic panning techniques to this pobem, we cuenty use a vey simpe symboic path panne, a vaiant on the one used o ou andmak-based navigation. The panne uses A* seach in the augmented topoogica map to ind a path to the goa. It uses this pan skeeton to assign peeed headings to the edges and nodes in the map, based on the expected tota tave distance to the goa and estimates about how ong it takes to tun. Diectives ae then associated with the Makov states: a go owad diective is assigned to each state whose oientation is the same as the peeed heading o its associated topoogica entity. The emaining states ae assigned actions that wi tun the obot towads the desied heading. Finay, a stop diective is assigned to the goa state and to neaby states (which heps to incease the tota pobabiity mass o the stop diective when the obot eaches the goa). Ou panne, and the voting heuistics used in action seection, ae ceay ineio compaed to optima POMDP soutions (in which diectives ae assigned to pobabiity distibutions, athe than individua states). Fo exampe, unike POMDP agoithms, ou panne cannot decide to take actions whose ony pupose is to gathe inomation. Sometimes, howeve, it can be advantageous to gathe additiona inomation that heps the obot to educe positiona uncetainty, even i that equies it to move away om the goa tempoaiy. At pesent, howeve, it is ineasibe to detemine even appoximate POMDP soutions given the size o ou state spaces and ou ea-time constaints [Lovejoy, 1991]. [Cassanda et a., 1994], o instance, epot that thei POMDP method can sove a pobem with 23 states in unde ha an hou, whie the mode o just ha o one oo o ou buiding has ove 3000 states. We sti intend to expoe POMDP agoithms, howeve, given ecent advances in appoximate agoithms [Pa and Russe, 1995] and the hope that the esticted topoogy o ou Makov modes might make them moe amenabe to eicient soutions. 5 Expeiments Whie Makov modes ae expessive and eativey eicient, they make stong independence assumptions. Empiica evidence is needed to detemine whethe, in this case, the Makov assumption is satisied we enough to yied good, eiabe navigation peomance. In this section, we epot on expeiments in two envionments o which the Makov assumption is ony an appoximation: a eaistic simuation o a pototypica oice coido envionment, and an actua mobie obot navigating in ou buiding. The same navigation code is used o both sets o expeiments, since the simuato and the obot have the exact same inteaces.

7 ... K C B J F G D E stat 2 goa 1 I H... goa 2 A... 5 metes stat 1 best-action best-above-theshod path eq. time speed eq. time speed s cm/s s cm/s JFI JFGFI JFGHGFI JFGFGFI JFGDGFI N/A Tabe 2: Expeiment 2 Figue 7: An Oice Coido Envionment best-action best-above-theshod path eq. time speed eq. time speed s cm/s s cm/s ABE ABCDE ABCKCDE N/A Tabe 1: Expeiment Expeiments with the Simuato Two navigation expeiments wee peomed with the obot simuato in the coido envionment shown in Figue 7. The topoogica map has 17 nodes and 36 diected edges. We modeed the uncetainty o the ength o a topoogica edge as a uniom distibution ove the inteva anging om 80 to 150 pecent o the ea ength o the edge. The esuting Makov mode has 1184 Makov states. The initia positiona uncetainty o both expeiments is minima: the initia pobabiity o the obot s actua ocation is about 90 pecent. The emaining pobabiity mass is distibuted in the vicinity o the actua ocation. In the ist expeiment, the task was to navigate om & to C. The peeed headings assigned by ou panne ae shown with soid aows. Note that the peeed heading between B and C is towads C because, even though the goa distance is a bit onge, this way the obot does not have to tun aound i it oveshoots B. We an a tota o 15 tias o both the best-action and the best-above-theshod stategies, a o which wee competed successuy (Tabe 1). The obot has to tave a athe ong distance om & beoe its ist tun. Since this distance is uncetain and coido openings ae occasionay missed, the obot occasionay oveshoots B, and then becomes uncetain whethe it is eay at C o B. Howeve, since the same diective is assigned to both nodes, this ambiguity does not need to be esoved: the obot tuns et in both cases and then goes staight. The same thing happens when it gets to D, since it thinks it may be at eithe D o E. The obot eventuay coects its beies when, ate tuning et and taveing owad, and it detects an opening to its et. At this point, the obot becomes aiy cetain that it is at E. A puey andmak-based navigation technique can easiy get conused in this situation, since it has no expectations about seeing this opening, and can ony attibute it to senso eo (which, in this case, is incoect). In the second expeiment, the obot had to navigate om & to Figue 8: Wean Ha at CMU. The peeed headings o this task ae shown with dashed aows. Again, we an 15 tias o both action seection stategies (Tabe 2). Fo easons that ae simia to those in the ist expeiment, the obot can conuse G with F. I it is at G but thinks it is pobaby at F, it tuns ight and goes owad. Howeve, when it detects the end o the coido but does not detect a ight coido opening, it eaizes that it must be at H athe than I. Since the pobabiity mass has now shited, it tuns aound and goes ove G, F, and I to the goa. This shows that ou navigation technique can gaceuy ecove om misjudgements based on wong senso epots even i it takes some time to coect its beies. It is impotant to eaize that this behavio is not tiggeed by any expicit exception mechanism, but esuts automaticay om the way the position estimation and action seection inteact. 5.2 Expeiments with Xavie Xavie, ou indoo mobie obot, is buit on an RWI B24 base and incudes bump sensos, sonas, a ase ange senso, and a coo camea on a pan-tit head. Conto, peception and panning ae a caied out on two on-boad, muti-pocessing 486-based machines. As mentioned, the pobabiistic navigation system uses a modiied vesion o the panne and essentiay the same obot contoe and senso intepetation components as ou andmak-based navigation system. Thus, dieences in peomance can be diecty attibuted to the dieent navigation appoaches. In addition, to aciitate compaisons we an Xavie aong the same outes as epoted in [Simmons, 1994]. In paticua, the obot tavesed om point S to G and back again (Figue 8) in some tias (45 metes each way), and in some cicumnavigated aound the buiding (150 metes). The topoogica map used to epesent the coidos in Figue 8 has 95 nodes and 180 diected edges. As with the simuato tias, the edge engths anged uniomy om 80 to S G

8 150 pecent o the ea coido ength. The esuting Makov mode has 3348 states. In 25 tias (mosty back and oth between points S and G), the obot successuy eached its goa in 22 cases, aveaging 30 cm/s whie tavesing a tota o ove a kiomete. In two o those cases, the obot missed seeing a junction, but tuned back when it eaized it had pobaby gone too a, and successuy continued. This success ate o 88% compaes avoaby with the 80% ate epoted in [Simmons, 1994]. The main dieence is that the pobabiistic navigation scheme uses a avaiabe senso inomation to hep ocaize itse. Fo exampe, whie the pobabiistic navigation uses the obot s dead-eckoning to diecty constain its position estimates, the andmak-based navigation uses metic inomation in ony two ways: it ignoes andmaks that ae epoted beoe a minimum distance has been taveed, and tuns aound ate a maximum distance. Simiay, the pobabiistic navigation scheme utiizes a senso epots, whie the andmak-based scheme pays attention ony to those eatues that might coespond to the expected andmak. One eect o this is that the pobabiistic navigation scheme tends to tun the obot eaie when enteing a junction: it oten gets enough conidence om a singe senso epot, whie the andmak-based scheme needs sevea (e.g., seeing both an opening to the side and the end o the coido ahead) beoe it decides to tun. The ew emaining aiues ae attibutabe to two souces: Occasionay the action seection heuistics ente a imit cyce and continuay tun the obot (we suspect this is due to a sotwae bug). Moe undamenta is that the oca obstace avoidance wi, especiay in oyes, move the obot a signiicant distance othogonay to its commanded heading. Since this is not cuenty epoted, the obot s position estimation becomes vey inaccuate. We can emedy this by epoting side motions and adding a side action to the Makov mode that wi cause the appopiate state tansitions. 6 Futue Wok and Concusions This pape has pesented ou ist eots at using patiay obsevabe Makov modes (POMDPs) o autonomous oice navigation. The appoach enabes a obot to utiize a its senso inomation, both positiona and eatue-based, in ode to obusty tack its ocation. A simpe path panne and action seection heuistics ae used to diect the obot s goa heading. Advantages o this appoach incude the abiity to account o uncetainty in the obot s initia position, actuato uncetainty, senso noise, and uncetainty in the intepetation o the senso data. Aso, by integating topoogica and metic inomation, the appoach easiy deas with uncetainty aising om incompete desciptions o the envionment. We ae extending this wok in sevea diections. We have impemented methods, based on EM eaning techniques, that passivey eine metic map inomation as we as the senso and action modes, and wi be testing it with Xavie. In addition, we ae deveoping impoved eaning techniques that ae moe esistant to vioations o the Makov assumption. We intend to pusue panning and action seection agoithms that appoximate optima POMDP poicies, and to compae thei peomance to the geedy heuistics descibed hee. Finay, we intend to add new souces o senso inomation, pimaiy vision-based eatue detectos. The impemented pobabiistic navigation system has demonstated its eiabiity, both in simuation and on Xavie, even in the ace o signiicant uncetainty. We beieve that such pobabiistic navigation techniques hod geat pomise o getting obots eiabe enough to opeate unattended o ong peiods o time in compex, uncetain envionments. Acknowedgements Thanks to Lonnie Chisman, Richad Goodwin and Joseph O Suivan o heping to impement pats o the navigation system and o many vauabe discussions. Swantje Wims heped peom some o the expeiments with the simuato. Reeences [Akin, 1987] R.C. Akin. Moto schema based navigation o a mobie obot: An appoach to pogamming by behavio. In Poceedings o the IEEE Intenationa Coneence on Robotics and Automation, pages , [Cassanda et a., 1994] A.R. Cassanda, L.P. Kaebing, and M.L. Littman. Acting optimay in patiay obsevabe stochastic domains. In Poceedings o the AAAI, pages , [Chisman, 1992] L. Chisman. Reinocement eaning with peceptua aiasing: The peceptua distanctions appoach. In Poceedings o the AAAI, pages , [Dean et a., 1993] T. Dean, L.P. Kaebing, J. Kiman, and A. Nichoson. Panning with deadines in stochastic domains. In Poceedings o the AAAI, pages , [Ees, 1989] A. Ees. Using occupancy gids o mobie obot peception and navigation. IEEE Compute, pages 46 57, [Goodwin, 1994] R. Goodwin. Reasoning about what to pan. In Poceedings o the AAAI, page 1450, [Hu and Bady, 1994] H. Hu and J.M. Bady. A Bayesian appoach to ea-time obstace avoidance o a mobie obot. Autonomous Robots, 1(1):69 92, [Kiman et a., 1991] J. Kiman, K. Basye, and T. Dean. Senso abstactions o conto o navigation. In Poceedings o the IEEE Intenationa Coneence on Robotics and Automation, pages , [Koenig and Simmons, 1994] S. Koenig and R. Simmons. How to make eactive pannes isk-sensitive. In Poceedings o the Intenationa Coneence on Atiicia Inteigence Panning Systems, pages , [Konoige, 1994] K. Konoige. Designing the 1993 obot competition. AI Magazine, 15(1):57 62, [Kotenkamp and Weymouth, 1994] D. Kotenkamp and T. Weymouth. Topoogica mapping o mobie obots using a combination o sona and vision sensing. In Poceedings o the AAAI, pages , 1994.

9 [Kosake and Kak, 1992] A. Kosake and A. Kak. Fast vision-guided mobie obot navigation using mode-based easoning and pediction o uncetainties. In Poceedings o the IEEE Intenationa Wokshop on Inteigent Robots and Systems, pages , [Kuipes and Byun, 1988] B.J. Kuipes and Y.-T. Byun. A obust, quaitative method o obot spatia eaning. In Poceedings o the AAAI, pages , [Lovejoy, 1991] W.S. Lovejoy. A suvey o agoithmic methods o patiay obseved Makov decision pocesses. Annas o Opeations Reseach, 28(1):47 65, [Mataic, 1991] M.J. Mataic. A distibuted mode o mobie obot envionment-eaning and navigation. Technica Repot TR 1228, Atiicia Inteigence Laboatoy, Massachusetts Institute o Technoogy, [Nichoson and Bady, 1994] A.E. Nichoson and J.M. Bady. Dynamic beie netwoks o discete monitoing. IEEE Tansactions on Systems, Man, and Cybenetics, 24(11): , [Noubakhsh et a., 1995] I. Noubakhsh, R. Powes, and S. Bichied. Devish: An oice-navigating obot. AI Magazine, 16(2):53 60, [Pa and Russe, 1995] R. Pa and S. Russe. Appoximating optima poicies o patiay obsevabe stochastic domains. In Poceedings o the IJCAI, pages , [Simmons, 1994] R. Simmons. Becoming inceasingy eiabe. In Poceedings o the Intenationa Coneence on Atiicia Inteigence Panning Systems, pages , [Simmons, 1995] R. Simmons. The 1994 AAAI obot competition and exhibition. AI Magazine, (in pess). [Smith and Cheeseman, 1986] R. Smith and P. Cheeseman. On the epesentation and estimation o spatia uncetainty. The Intenationa Jouna o Robotics Reseach, 5:56 68, [Tenenbeg et a., 1992] J. Tenenbeg, J. Kasson, and S. Whitehead. Leaning via task decomposition. In Poceedings o the Coneence Fom Animas to Animats, pages , 1992.