The Best of Times and the Worst of Times are Interchangeable

Transcription

1 The Best of Ties and the Worst of Ties are Interhangeale Guy E. Hawkins a, A.A.J. Marley,, Andrew Heathote d, Terry N. Flynn, Jordan J. Louiere, and Sott D. Brown d a Shool of Psyhology, Uniersity of New South Wales, Australia Deartent of Psyhology, Uniersity of Vitoria, Canada Centre for the Study of Choie, Uniersity of Tehnology, Sydney, Australia d Shool of Psyhology, Uniersity of Newastle, Australia Astrat We oonly deterine the ost referred (est) and least referred (worst) of a set of otions, yet it is unlear whether the two hoies are ased on the sae or different inforation. We exained est and worst hoies using disrete hoie tasks, where artiiants seleted either the est otion fro a set, the worst otion, or seleted oth the est and the worst otion. One exerient used eretual judgents of area, and another used onsuer referenes for arious attriutes of oile hones. In oth doains, we found that the task (est, worst, or est and worst) does not alter the referenes exressed for the est (resetiely, the worst) otion. We also osered that the hoie roailities were onsistent with a single latent diension otions that were frequently seleted as est were infrequently seleted as worst, and ie ersa oth within and etween resondents. A quantitatie odel of hoie and resonse tie roided onergent eidene on those relations, with odel ariants that assued an inerse relationshi etween the estiated araeters for est and worst hoies aounting well for the data. We onlude that the dierse tyes of est and worst hoies that we studied an e oneied as oosing ends of a single ontinuous diension rather than distint latent entities. We disuss these results in the light of rather different results for aeting (e.g., urhasing) and rejeting (e.g., not urhasing) otions fro a set. Keywords: Deision-aking; est hoie; worst hoie; est-worst saling; statetrae analysis; atheatial odel; resonse tie. Corresondene onerning this artile ay e addressed to: Guy Hawkins, Shool of Psyhology, Uniersity of New South Wales, Sydney NSW 2052, Australia; Eail: guy.e.hawkins@gail.o.

2 DIMENSIONALITY OF BEST AND WORST CHOICES 2 Introdution Suose you seek to urhase a diaond ring for your loed one and the loal jeweler has fie in stok. The aailale otions differ aross ultile, indeendent attriutes, suh as size, ut, larity, and olor. When faed with this olex task, how do you ake a deision? For exale, you ould eliinate y asets until a single otion reains (Tersky, 972), or a suset of the fie otions ould e generated y inluding the ost desirale rings, and then rejeting the least faorale of the suset until left with a single otion. As a result of arious suh deision rules, you ight end u knowing whih is your ost referred (est) and least referred (worst) of the aailale otions; for reity and in agreeent with the releant literature, ost of the tie in the following we refer to these as the est and the worst otion. Then you ight aet (is., urhase) the est aailale otion, ut you ight not urhase (is., ight rejet) it eause all the rings are too exensie. This illustration shows that est and/or worst hoies do not gie the sae inforation as aet and/or rejet deisions. Here, we inestigate whether oth est and worst hoies an e exlained y a single latent onstrut or reresentation, and hene ake use of the sae underlying inforation. We also exaine whether the seletion of the worst otion in a set influenes whih otion is seleted as est, and siilarly whether the seletion of the est otion in a set influenes whih otion is seleted as worst; these are different tests of whether the two tyes of judgent interat. Before roeeding to our study of est and/or worst hoies, we riefly disuss the literature on aet and/or rejet deisions to further suort our stateent that they differ; we return to this toi in our Results and Disussion of Exerient. Soeties we write est hoie, worst hoie, or est-worst hoie as generi ters. Best-worst hoie has een extensiely studied in disrete hoie exerients siilar to our own (e.g., Finn & Louiere, 992; Marley & Pihlens, 202). Aeting and rejeting deisions hae een studied ainly fro the ersetie of fraing effets in judgent. In this aradig, deision tasks are designed in suh a way that they an lead to inonsistent referenes under different exeriental onditions. For exale, Shafir (993) resented hyothetial award (aet) and deny (rejet) hoies regarding the two arents in an only-hild-sole-ustody ase. The hoie inoled one ioerished low ariane otion (Parent A), with ostly aerage attriutes; and one enrihed high ariane otion (Parent B) with soe exellent and soe awful attriutes; two sets of resondents saw the sae two otions, ut were gien different instrutions. Resondents asked to whih arent they would award sole ustody tended to aet the enrihed otion, yet resondents asked to whih arent they would deny sole ustody again tended to selet the enrihed otion. Siilar atterns of referenes hae een osered in inestent deisions (Cheng & Chiou, 200), hoies aong jo andidates (Ganzah, 995), and een in a syhohysial judgent task (Tsetsos, Chater, & Usher, 202). Suh reersals ight arise eause resondents fous on different attriutes when aeting and rejeting (e.g., an Buiten & Keren, 2009), or erhas the reersals are ediated y the aount of elaoration of attriute inforation (Ganzah & Shul, 995; Juliano & Wilox, 20). The statistial rinile oonly used in this researh is the logi of dissoiations: if one exeriental aniulation (e.g., high s. low ariane hoie otions) has different effets under different leels of another aniulation (e.g., deisions to aet s. rejet), we ight onlude that

3 DIMENSIONALITY OF BEST AND WORST CHOICES 3 two different ognitie roesses are at work (e.g., different ognitions aout aeting s. rejeting). Although this logi is oon in syhology, and an aear oelling, it an also e isleading (Loftus, 978; Wagenakers, Kryotos, Criss, & Ierson, 202). Also, any of the data are fro etween-sujet designs. Rather than searhing for differenes or inonsistenies etween aeted and rejeted otions, here we aied to deterine whether hoies ade under a single instrution set naely ost referred (est) and least referred (worst) an e exlained with reourse to only a single latent ariale, suh as the utility (or alene) of different attriutes and otions. For instane, when asked to sequentially onsider a series of ultiattriute otions suh as autooiles or jo andidates, resondents tend to arrie at the sae final referene (the first of following three studies used aet/rejet, the others inlude/exlude; Huer, Neale, & Northraft, 987; Lein, Jaser, & Fores, 998; Lein, Prosansky, Heller, & Brunik, 200). Howeer, exliit instrution to onsider external fators, suh as seletion-related osts in ersonnel seletion ay result in different hoie outoes (Huer et al., 987). We wondered whether, in the asene of aniulations designed to lead the to do otherwise, eole ight ase est and worst hoies on the sae ognitie inforation. Seifially, we addressed two questions related to suh hoies. Firstly, we exained whether seleting oth the est and worst otion gies different data for est (resetiely, worst) than when only a est (resetiely, worst) hoie is ade. We inestigated this question y testing three grous of artiiants with idential hoie sets. One grou was asked only to selet their ost referred (est) otion, another grou was asked only to selet their least referred (worst) otion, and the third grou was asked to selet oth their ost and their least referred otion. Our seond question addressed whether est and worst hoies are onsistent with a single underlying diension (latent ariale). That is, eah of the releant araeters underlying a est hoie is a onotoni (dereasing) transforation of eah orresonding alue in a worst hoie (oth relatie to other aailale otions); or are est (resetiely, worst) hoies ased on two (or ore) distint latent ariales? We exained this question y analysis of data fro the grou that ade est and worst hoies (a within-sujets oarison), and oaring the data fro the grou that ade est hoies, only, with the data for the grou that ade worst hoies, only (a etween-sujets oarison). We erfored two exerients that used est-worst saling. In Exerient we asked for eretual judgents aout area, and in Exerient 2 we assessed referenes for attriutes of oile hones. Exerient : Peretual Choie In Exerient, artiiants were resented with three shaded retangles on eah trial, eah with a different area. Partiiants were asked to selet the retangle with the largest area (as an analog to est hoie) in the est-only ondition, selet the retangle with the sallest area (as an analog to worst hoie) in the worst-only ondition, or selet the retangle with the largest and the retangle with the sallest area in the est-worst ondition. Although it ay at first see extreely unlikely that ontext effets, or differenes etween est (e.g., largest) and worst (e.g., sallest) hoies, ould arise in sile eretual hoie aradigs, there is at least one reent study that has identified just suh effets (Tsetsos et al., 202). To roide a sile exale of the kind of ontext effet that

4 DIMENSIONALITY OF BEST AND WORST CHOICES 4 ould lead to different est and worst roesses, artiiants ight hae weighted the height of retangles ore heaily than the width of retangles when aking largest hoies, ut ie ersa for sallest hoies. In fat, any deision roess wherey arious asets of the stiuli hae different influene under est and worst hoies has the otential to rejet our null hyothesis at the leel of otions, if not at the leel of asets (attriutes). Partiiants Sixty-nine first-year syhology students fro the Uniersity of Newastle artiiated in the exerient online in exhange for ourse redit, and were randoly alloated to either the est-only (n = 23), worst-only (n = 24) or est-worst (n = 22) onditions. Materials and Methods The eretual stiuli were adated fro Truelood, Brown, Heathote, and Buseeyer (203) and Hawkins et al. (in ress). We fatorially rossed three retangle widths (55, 6, 67 ixels) with three heights (0, 2, 33 ixels) to reate nine unique retangular stiuli ranging in area fro ixels. The stiulus set generated a range in diffiulty fro sile judgents, with easily differentiale stiuli at the extrees of the set (e.g., 6050 fro 89), to diffiult judgents etween stiuli at one end of the range (e.g., 7370 fro 738). On eah trial, three of the nine retangles were randoly saled without relaeent. The stiuli were resented at the ertial enter of the dislay, sidey-side in a horizontally entered row, as shown in Figure. All retangles were sujet to rando ertial offset (±25 ixels) to reent the use of alignent ues in judging area. Partiiants in eah ondition oleted six loks of 00 trials. Figure. Illustratie exale of a trial fro the est-worst ondition in Exerient. Partiiants in the est-only ondition were asked to identify the retangle with the largest area. Partiiants in the worst-only ondition were asked to identify the retangle

5 DIMENSIONALITY OF BEST AND WORST CHOICES 5 with the sallest area. Partiiants in the est-worst ondition were asked to identify the retangle with the largest area and that with the sallest area, and were restrited fro seleting the sae retangle as oth the largest and sallest otion. All resonses were reorded with a ouse lik, and in the est-worst ondition the artiiant was free to selet the ordering of the two resonses; the est resonse ould e ade first and the worst resonse seond, or ie ersa. We reorded artiiants hoies, and the tie required to ake the, though we restrit our riary analyses to the hoie data. Analyti Aroah We had two riary ais: to deterine whether the at of hoosing a worst otion alters the est hoies (and ie ersa), and also to deterine whether est and worst hoies arise fro a single latent ariale. These researh questions do not naturally lend theseles to the traditional null hyothesis statistial testing (NHST) fraework. For onsisteny with reious researh we reort NHST analyses, ut our riary onlusions are drawn fro Bayesian analysis and non-araetri state-trae analysis (Baer, 979). Bayesian Analysis. We ondut all Bayesian analyses using the ethods of Morey and Rouder (203), who ileented oon linear odels suh as ANOVA and regression within a Bayesian fraework. Morey and Rouder s aroah rodues Bayes fators for linear odel effets (like those in ANOVA) indiating the weight of eidene for or against artiular hyotheses. We use a targeted ersion of this aroah to test the effets of interest to our hyotheses. Firstly, we test if there is eidene for the odel with a ain effet of attriute (retangle area), using the Bayes fator BF A. More iortantly for our hyotheses, we oare this ain effet odel against a odel with the sae ain effet of attriute as well as an interation etween attriute and ondition (est-only s. est in est-worst, worst-only s. worst in est-worst, est s. worst), and reort this as a ratio of Bayes fators: BF A+A C. Values of this ratio greater than one roide eidene A for the inlusion of the interation ter in addition to the ain effet of attriute, whereas alues less than one roide eidene for exlusion of the interation (i.e., for the ain effet only odel). Note that, unlike NHST, whih annot aet a null hyothesis, the Bayesian analysis is equally ale to find eidene for the siler (ain effet only) or for the ore olex (ain effet and interation) hyotheses. State-Trae Analysis. State-trae analysis is designed to answer questions suh as an the osered data e exlained y a single latent ariale? The analysis is nonaraetri, assuing only that the latent ariales (suh as utility) are onotonially related to osered ariales (suh as hoie roaility). It is designed to oeroe issues of sale-deendene in the analysis of range-restrited deendent ariales, whih an gie rise to surious dissoiations. In artiular, the existene of searate ognitie systes is often inferred fro a dissoiation. The dissoiation is easured y an interation, where the effet of one aniulation deends on the other. Howeer, inferring the existene of suh interations is ade diffiult y the ossiility of sale deendene. For exale, if one ondition leads to near-eiling or near-floor erforane, the effet of the other aniulation Morey and Rouder (203) roide sile ethods to ileent the Bayesian analyses desried in this anusrit in the freely aailale R rograing enironent (R Deeloent Core Tea, 203).

6 DIMENSIONALITY OF BEST AND WORST CHOICES 6 will e fored to zero, leading to a surious interation. There are any other ways that sale deendene an lead to isleading interations; aarent interations that do not really indiate the resene of an underlying dissoiation (see, e.g., Baer, 979; Dunn & Kirsner, 988). This kind of role an e artiularly diffiult in exerients on hoie roortions (suh as Shafir, 993; Tsetsos et al., 202) eause of the tight ounds on the deendent ariales. State-trae analysis oeroes this shortoing y relaxing assutions aout the easureent sales it assues only a onotoni relationshi etween the latent and osered ariales. The analysis hinges on a state-trae lot, in whih different deendent ariales are lotted against one another. If the oints of the lot an e onneted with a single, onotoni (i.e., always inreasing or dereasing) ure, then the data are onsistent with a single latent ariale (for atheatial details see Baer, 979). Drawing inferenes aout onotoniity in state-trae analysis is a diffiult statistial role. There has een reent rogress using araetri ootstraing (Dunn, Newell, & Kalish, 202; Newell & Dunn, 2008; Newell, Dunn, & Kalish, 200) and Bayesian aroahes with order onstrained hyotheses (Prine, Brown, & Heathote, 202). Although these aroahes are roising and eah hae erit, we do not ileent the here due to our oination of within- and etween-sujets easures, and the high diensionality of the stiuli used in Exerient 2. Rather, we use a ore onentional aroah ased on onfidene interals (e.g., Busey, Tunniliff, Loftus, & Loftus, 2000; Loftus, Oerg, & Dillon, 2004). In artiular, we onstrut least signifiant differene (Saille, 2003) ellises around data oints on a state-trae grah, with these ellises ased on Morey s (2008) odifiation of Loftus and Masson s (994) within-sujet standard errors for within-sujets oarisons. If all ellitial regions in the state-sae an e onneted with a single onotoni funtion we are unale to rejet the null hyothesis that the data an e exlained y a single latent diension. Results and Disussion We used the resonse tie inforation to sreen data in a two-stage roess on the asis of unusually fast or slow resonses. First, we arked as extree outliers and exluded fro analysis any resonses that were faster than.2 seonds or slower than 20 seonds (2.33% of total trials), and further exluded all data fro one artiiant who had ore than 20% of their resonses arked as extree outliers. Seond, for the reaining 68 artiiants we alulated resonse tie eans and standard deiations on an indiidual artiiant asis, and exluded as outliers any trials with resonse ties at least three standard deiations greater than the artiiant s ean resonse tie. Coined aross oth riteria, 4.07% of trials were onsidered outliers and reoed fro analyses. Our analyses are ased on the roortion of ties that eah of the nine retangles was seleted as the largest otion (est hoie roortions) for artiiants in the est-only and est-worst onditions, and the orresonding roortion for sallest hoies (worst hoie roortions) for artiiants in the worst-only and est-worst onditions. For exale, if a artiular retangle was inluded in the hoie sets of 200 trials and was seleted as the largest retangle in 80 of those trials and as the sallest retangle in 0 of those trials, then the orresonding est and worst hoie roortions are 80/200 =.4 and 0/200 =

7 DIMENSIONALITY OF BEST AND WORST CHOICES 7.05, resetiely. The hoie roortions for the nine retangular stiuli were alulated searately for eah artiiant then aggregated oer artiiants for analysis. Sine our hoie sets were randoly generated on eery trial, we ade a further hek to exaine how often there were hoie sets with doinated or doinating otions. A hoie set has a doinating (resetiely, doinated) otion when one retangle has greater (resetiely, saller) width and height than the other two retangles. If artiiants attended to the aniulation of area, then a doinating otion should e relialy hosen as est and rarely hosen as worst, and ie ersa for doinated otions. Doinating alternaties aeared in 9.6% of trials. As exeted, when a doinating otion was aailale it was seleted as the largest retangle on any trials (est-only 90.%, est-worst 90.3%) and seleted as the sallest retangle on only a sall roortion of trials (worstonly 0.9%, est-worst 4.%). Siilarly, doinated alternaties aeared in 9.5% of trials and were rarely seleted as the largest otion (est-only 3.7%, est-worst 5.%) ut frequently seleted as the sallest retangle (worst-only 96.4%, est-worst 86.2%). These hoie roortions suggest that artiiants were aware of, and resonsie to, our aniulation of area. Best Choies are Unaffeted y Worst Choies, and Vie Versa We first heked whether the additional at of seleting the worst otion fro a set of otions altered seletion of the est otion. We exained the est hoie roortions with a two fator ixed ANOVA: 2 ondition (etween: est-only, est-worst) 9 retangle areas, using Greenhouse Geisser adjusted degrees of freedo where aroriate. There was a strong ain effet of area on judgents, where larger retangles were hosen ore often as largest (est) than were saller retangles, F (2., 89.3) = 09.7, <.00, BF A > In ontrast, ondition est-only ersus est in est-worst had no statistially reliale effet on the roortion of est hoies in the interation effet, F (2., 89.3) =.4, =.88, reresented in the Bayesian analysis as eidene in faor of the ain effet odel oer the odel that inludes an interation effet, BF A+A C A =.007. This Bayes fator indiates that the odel with no interation is oer 40 ties (i.e., /.007) ore likely than the odel with an interation. We also onduted the orresonding analysis for worst hoie roortions: whether the additional at of seleting the est otion fro a set of otions altered seletion of the worst otion. The riary result for the roortion of worst hoies was in the sae diretion as for est hoies, though the eidene was not as strong. There was again a strong ain effet of area, where saller retangles were ore often hosen as sallest (worst) than were larger retangles, F (2.3, 97.8) = 6, <.00, BF A > Siilar to est hoie roortions, ondition worst-only ersus worst in est-worst did not hae a statistially reliale interation effet on area, F (2., 89.3) =.9, =.5. Bayesian analysis indiated slight referene for the odel with only a ain effet oer the interation odel, BF A+A C A =.84. We used state-trae analysis to test ore diretly whether est hoies were unaffeted y the at of aking worst hoies. The left anel of Figure 2 shows a state-trae lot of est hoie roortions for eah retangle area fro the est-only ondition (y-axis) and the est-worst ondition (x-axis). The oerlaid line in the left anel of Figure 2 deonstrates that a single, onotonially inreasing ure onnets all data oints in the state-sae,

8 DIMENSIONALITY OF BEST AND WORST CHOICES 8 eaning that the data are onsistent with an exlanation ased on a single latent ariale that seletion of the est otion fro a set is not affeted when one also onsiders the worst otion. Note that the onfidene regions are quite sall, indiating that a lak of ower was unlikely to e the ause of this finding. For reity we hae not inluded in the ain text the orresonding state-trae lot for worst hoie roortions etween the worst-only and est-worst onditions. This analysis led to the sae outoe as the reious analysis, with a siilar lot as the left anel of Figure 2; the releant figure is in online suleentary aterial. Best Choie Proortions (Best Only Condition) Area (ixels) Best Choie Proortions (Best Worst Condition) Worst Choie Proortions (Worst only Condition) Figure 2. State-trae lots of est and worst hoie roortions fro seleted onditions in Exerient. The y-axes lot est hoie roortions fro the est-only ondition (oth anels). The x-axes lot est hoie roortions fro the est-worst ondition (left anel) and worst hoie roortions fro the worst-only ondition (right anel). The syols laeled 9 reresent areas of the nine unique retangles (in ixels). Ellises reresent etween-sujets least signifiant differenes. The lak ure reresents a onotoni ure joining all lots in eah anel inreasing in the left anel, dereasing in the right anel onsistent with the interretation of a data generating roess with a single latent ariale. Best Choies are Monotonially Related to Worst Choies Our seond analysis had two arts. We first erfored a within-sujets analysis foused solely on the est-worst ondition, oaring hoie roortions for the largest and sallest otions, and seondly as a etween-sujets analyses oaring hoie roortions fro the est-only and worst-only onditions. As exeted, retangles ost often hosen as the largest were also least often seleted as the sallest, and ie ersa. Two-way reeated-easures ANOVA indiated a strong interation etween est and worst hoie roortions and retangle area in the est-worst ondition, and siilarly for two-way ixed

9 DIMENSIONALITY OF BEST AND WORST CHOICES 9 ANOVA etween the est-only and worst-only onditions, F (.7, 36) = 4, <.00, and F (2, 86.6) = 267, <.00, resetiely. Siilarly, the Bayesian analysis indiated ery strong suort for the interation effet oer-and-aoe the ain effet of retangle area in oth analyses, BF A+A C A > and BF A+A C A > , resetiely. These results suggest that artiiants were autely sensitie to retangle area, learly disriinating retangles with large and sall area, regardless of whether those judgents were ade within or aross artiiants. Diret eidene that est and worst hoies are ased on the sae latent ariale oes fro state-trae analysis. The state-trae lot in the right anel of Figure 2 lots est hoie roortions fro the est-only ondition against the worst hoie roortions fro the worst-only ondition, as a funtion of retangle area. As with the reious statetrae analysis, all oints in the state-sae an e onneted with a single onotoni ure (dereasing in this ase). This suggests that seletion of the largest and sallest otions is onsistent with an exlanation ased on a single latent ariale, and that different roesses need not e inoked for est and worst hoies in this ase. 2 The right anel in Figure 2 reresents a ore stringent test of the hyothesis that est and worst hoies are ased on the sae inforation, eause that test inoles oarison aross indeendent sujets (est-only and worst-only onditions). The within-sujets analysis roided een stronger suort for our thesis (see suleentary online aterial). Exerient suorted the hyothesis that est hoies are not influened y the at of aking a worst hoie. We also suorted the hyothesis that est and worst hoies are ade on the sae ases. It ould e argued, howeer, that these results are unsurrising gien the task at hand: if resondents aid attention solely to retangle area, as instruted, then there is no reason to exet anything other than a single latent ariale. Neertheless, syhologial siene is relete with aarently sile tasks whih lead to olex, suotial, or irrational strategies. As a lausile hyothetial exale, if judgents of retangle area were influened y the aset ratio in addition to area, then the data ight hae differed to what we osered. For exale, three retangles with the sae area ut different aset ratios ight rodue different hoie roortions, and it is ossile that aset ratio ight e differentially iortant for largest ersus sallest judgents (height ight e the ost salient diension for seleting the largest area, ut not neessarily for seleting the sallest area). Neertheless, in Exerient 2 we ounter the ossile ritiis y reliating all key results fro Exerient using olex, ulti-attriute stiuli in onsuer judgents; a aradig in whih referene reersals and aradoxial hoies are known to often our. Exerient 2: Consuer Choie In Exerient 2 we aied to reliate and extend the results of Exerient, oing fro a eretual hoie task to a onsuer judgent task. We again exained whether seleting a worst otion alters est hoies, and ie ersa, and whether est and worst hoies are onsistent with a single latent diension. In Exerient 2, a single data- 2 A searate state-trae analysis onduted on the oonent attriutes of the retangles width and height also suorted a single latent ariale for est and worst hoies at the leel of attriutes (figures not shown).

10 DIMENSIONALITY OF BEST AND WORST CHOICES 0 generating roess ight e utility, where the strength of referene for a rodut ight range on a single diension fro highly desirale (high utility) to highly undesirale (low utility). In Exerient 2 we asked artiiants to judge oile hones desried y quantitatie attriutes, suh as rie, aera quality, and eory aaity. As in Exerient, artiiants were randoly alloated to one of three etween-sujets onditions: artiiants either seleted their est hone fro eah hoie set (est-only), their worst hone fro eah hoie set (worst-only), or their est and their worst hone fro eah hoie set (est-worst). We exeted that the est hoie roortions fro the est-only and estworst onditions would e suh that the roortion of ties eah leel of eah attriute (attriute-leel, e.g., rie: $99, $99, $299, et.) was hosen as est 3 would e the sae in the two onditions; and that a arallel result would hold for worst hoie roortions. We also exeted that hones ore often seleted as est would e less often seleted as worst, and ie ersa, in a anner onsistent with a single latent diension, oth within the est-worst ondition, and etween the est-only and worst-only onditions. Partiiants Method Eighty-seen first-year syhology students fro the Uniersity of Newastle artiiated in Exerient 2 online in exhange for ourse redit, and were randoly alloated to either the est-only (n = 27), worst-only (n = 30), or est-worst (n = 30) ondition. Materials and Methods The stiuli were adated fro Marley and Pihlens (202) to reflet standards for urrent oile hones. Partiiants were asked to selet etween oile hones that aried on fie attriutes, with eah attriute haing three leels. Three oile hone rofiles were resented on eah trial, with eah hone haing a single leel fro eah of fie attriutes. Eery hone on eah trial was randoly generated fro the set of ossile hones. An exale hoie set is shown in Figure 3 and a full listing of the attriutes and the leels of eah attriute, whih we refer to as attriute-leels, is shown in Tale. The fie attriuteleels that ade u eah hone were randoly hosen on eah trial (hoie set). Partiiants in all three onditions oleted 50 hoie sets, or trials, in total; six loks eah of 25 trials. Partiiants in the est-only ondition were asked to selet their ost referred hone and artiiants in the worst-only ondition were asked to selet their least referred hone. Partiiants in the est-worst ondition were asked to selet oth their ost and least referred hone. As in Exerient, resonses were ade with a ouse lik and ould e roided in either order in the est-worst ondition est, then worst, or worst, then est. Partiiants were restrited fro seleting the sae hone as oth the ost and least referred otion. We reorded all hoies, and the ties taken to ake the. 3 An attriute-leel is hosen as est when a hone is seleted as est that inludes that attriute-leel.

11 DIMENSIONALITY OF BEST AND WORST CHOICES Figure 3. Illustratie exale of a hoie set fro the est-worst ondition in Exerient 2. Results and Disussion As in Exerient, we used resonse tie inforation to sreen data on the asis of unusually fast or slow resonses. We defined as extree outliers and exluded fro analysis any resonses that were faster than.35 seonds or slower than 80 seonds (2.83% of total trials), and again exluded any artiiant who had ore than 20% of their trials arked as outliers (three artiiants reoed). For the reaining 84 artiiants, we alulated eans and standard deiations of resonse ties on an indiidual artiiant asis, and exluded as outliers any trials with resonse ties that were ore than three standard deiations larger than a artiiant s ean resonse tie. In total, 4.7% of trials were onsidered fast or slow outliers and were reoed fro analyses. We alulated est hoie roortions for eah attriute-leel as the roortion of ties that the hone hosen as est ontained that attriute-leel, with the analogous easure for worst hoie roortions. For exale, if a artiular attriute-leel (say, $250) aeared in 30 hoie sets for a artiular artiiant, and was in the hone seleted as ost referred in 78 of those 30 hoie sets, then it has the est hoie roortion 78/30 =.6. As in Exerient, the est and worst hoie roortions were alulated on an indiidual artiiant asis and then aggregated oer artiiants for analysis.

12 DIMENSIONALITY OF BEST AND WORST CHOICES 2 Tale : The fie attriutes and their oined 5 leels used in Exerient 2, adated fro Marley & Pihlens (202). Attriute Attriute-leels Prie $250 $500 $750 Caera Video aaility Handset eory Battery life 2 egaixel aera 4 egaixel aera 8 egaixel aera None Standard definition High definition 8 GB 6 GB 32 GB 4 hrs talk tie 8 hrs talk tie 6 hrs talk tie We alulated searate two-way ANOVAs and Bayes fators for eah attriute. In the ANOVAs we oared attriute-leel rossed with ondition (est-only s. est in est-worst; worst-only s. worst in est-worst, et.) against a ain effet odel inluding just the attriute leels. For the ANOVAs, we adoted a Bonferroni-adjusted signifiane leel: α =.05/5 =.0. Best Choies are Unaffeted y Worst Choies, and Vie Versa We first onfired that the attriute-leels had a reliale effet on erforane followed y exaination of whether seleting a least referred otion altered the hoie of the ost referred otion, and ie ersa. Partiiants were attentie to the seifi features of hones, with strong differenes in referene for the different leels of eah attriute (ain effets). Phones were relialy ore often hosen as est (i.e., larger est hoie roortions; est-only and est-worst onditions) when they were heaer, had a etter aera and ideo aaility, larger handset eory and longer attery life, all F s > 20 and all s <.00 using Greenhouse-Geisser adjusted degrees of freedo, and all BF A s > 0 4. Siilarly, hones were relialy ore often hosen as worst (i.e., larger worst hoie roortions; worst-only and est-worst onditions) when they were ore exensie, had a oorer aera, no ideo aaility, saller handset eory and a shorter attery life, all F s > 6 and all s <.00, and all BF A s > 0 8. The uer anels of Figure 4 lot est hoie roortions for attriute-leels fro the est-only ondition against the equialent roortions alulated fro the est-worst ondition. 4 The least signifiant differene ellises for all lot oints fall on the diagonal 4 We do not show the orresonding lots for rofiles hones as we do for the retangles studied in Exerient. Sine we randoized the leels of eah attriute to eery hone, there exist any ossile

13 DIMENSIONALITY OF BEST AND WORST CHOICES 3 Best Choie Proortions (Best Only Condition) Prie 2 $250 2 $500 3 $ $250 2 $500 3 $750 2 Prie Caera Best Choie Proortions (Best Worst Condition) Caera Video None 2 SD 3 HD None 2 SD 3 HD Video Meory 2 3 8GB 2 6GB 3 32GB Meory 2 3 8GB 2 6GB 3 32GB Battery 2 3 4hr 2 8hr 3 6hr hr 2 8hr 3 6hr 2 Battery Worst Choie Proortions (Worst Only Condition) Figure 4. Attriute-leel hoie roortions for the oile hone stiuli in Exerient 2. The uer anels show the attriute-leel est hoie roortions for the est-only ondition (y-axes) against the attriute-leel est hoie roortions for the est-worst ondition (x-axes) for eah attriute, searately. The dashed gray lines show the y = x line where referenes would fall if they were identially distriuted aross the est-only and est-worst onditions. The lower anels show the attriute-leel est hoie roortions for the est-only ondition (y-axes) against the attriute-leel worst hoie roortions for the worst-only ondition (x-axes). The dashed gray lines show where the attriute-leel worst hoie roortions would fall if they were the reerse of the attriute-leel est hoie roortions (y = x). Ellises reresent etween-sujets least signifiant differenes. y = x line, indiating that est hoies were not altered y also requiring a worst hoie (the orresonding figure for worst hoie roortions is shown in the suleentary aterial). Suorting this, there were no reliale interation effets etween ondition est-only ersus est in est-worst and attriute, for any of the fie attriutes, all s >.3. The analogous Bayes fators indiated no suort for the interation effet oer and aoe the ain effet of attriute; eory BF A+A C A =.33, all other BF A+A C A s <.6. This suggests that onsidering the least referred otions did not alter the referene for the ost referred otions. A siilar result eerged for worst hoie roortions, though not as strong: all s >.06, and the Bayes fators for two leels roided soe eidene for the = 5.9 and alternatie hyothesis (i.e., the resene of an interation) rie BF A+A C A =.47 while the reaining Bayes fators roided weak suort for aera BF A+A C A the null; BF A+A C s =.3,.35, and.87 for the ideo, eory, and attery attriutes, resetiely. On alane, a oined Bayes fator assuing attriute indeendene A indiates suort for the null hyothesis of no interation, BF A+A C A =.28. hone rofiles suh that we ould not otain reliale hoie roortion data for any artiular rofile.

14 DIMENSIONALITY OF BEST AND WORST CHOICES 4 The results shown in Figure 4 strongly and unaiguously suort onotoniity and hene one latent diension within attriutes. Although this is a strong onlusion, the analyses shown in Figure 4 annot test whether all attriutes a through a single ( utility ) diension. To reedy this, we also alulated aross-attriute state-trae analyses, shown in Figure 5. These lots oare the attriute-leel est hoies fro the est-only and est-worst onditions (uer triangle anels), and also oare the attriute-leel worst hoies fro the worst-only and est-worst onditions (shown in suleentary aterial). Eah anel akes the oarison for a air of attriutes (suh as rie and aera). A onotoni line in a anel indiates strong eidene that hoies ased on those two attriutes use the sae inforation in the different onditions (est-only and est-worst, or worst-only and est-worst). The (x, y) oordinate of eah oint reresents the ean hoie roortion for a single attriute-leel for two onditions (e.g., est-only, est-worst), with ellises reresenting etween-sujets least signifiant differenes (Saille, 2003). For exale, the uer row shows the rie attriute, with the first anel oaring rie against the aera attriute, the seond anel oaring rie against the ideo attriute, and so on. A single onotonially inreasing line an onnet the ellises in all anels aoe the ain diagonal. This eans that all airs of attriutes roide eidene in faor of a single underlying deision ariale. This result erits a deisie onlusion aout diensionality: the ten airwise oarisons etween attriutes an e reresented with a single latent ariale. This onlusion is warranted gien the transitie logi of state-trae analysis: if A and B are onotonially related, and B and C are onotonially related, then A and C ust e onotonially related. Therefore, the joint reresentation of all fie attriutes is neessarily onotoni and arose fro a single latent ariale that is, seleting the worst otion fro the set of otions did not hange the asis on whih est hoies were ade. The orresonding analysis for worst hoie roortions in the worst-only and estworst onditions led to the sae onlusion, and is resented in suleentary aterial. Best Choies are Monotonially Related to Worst Choies We next exained the relationshi etween attriute-leel est and worst hoies. Reeated-easures ANOVA etween est or worst hoie and the attriutes indiated that the attriute-leels that aeared ost often in the referred hones also aeared least often in the hones hosen as worst, and ie ersa see lower anels of Figure 4 for est-only ersus worst-only (figures lotting est s. worst hoie roortions fro the est-worst ondition are shown in suleentary aterial). This effet was reliale in the interation etween est hoie against worst hoie and the attriute-leels for all fie attriutes, aross the est-only ersus worst-only and est ersus worst in the est-worst ondition, all F s > 20 and all s <.00, and in the analogous Bayes fators, all BF A+A C A > 0 5. These results suggest that artiiants relialy attended to the attriutes that orise oile hone quality, learly disriinating etween hones that hae ositie and negatie qualities (ost and least referred, resetiely). They also suggest that this attern holds regardless of whether artiiants ade oth est and worst hoies, or only one of the two. State-trae analysis roided strong eidene that a single latent diension exlains osered attriute-leel est and worst referenes in Exerient 2. In Figure 5 (lower triangle anels), the est and worst hoie data and their ellises of least signifiant differenes s

15 DIMENSIONALITY OF BEST AND WORST CHOICES 5 an e onneted in all anels with a onotonially dereasing line. This roides eidene that referenes for the est and worst attriute-leels are onsistent with an exlanation ased on a onotoni transforation of the sae underlying inforation oosing ends of a single ontinuu where est an e onsidered the oosite of worst, and ie ersa. This relationshi held whether artiiants seleted only their ost or least referred otion (Figure 5) or oth (suleentary aterial). Our state-trae analysis in Exerient 2 deonstrated a reiroal relationshi etween est and worst referenes at the leel of attriutes, ut we did not diretly test whether the orresonding reiroal relationshi held at the leel of otions (i.e., oile hones) as we did in Exerient. It was feasile to test the onotoniity of est and worst referenes at the leel of otions in Exerient as there were only 9 unique retangle areas ut not in Exerient 2 due to the large nuer of otions (243): eause of this large nuer of otions, we had insuffiient data to diretly estiate the est and worst drift rate for eah of the. In Exerient 2 our finding of a reiroal relationshi etween est and worst referenes at the leel of attriutes suorts an exlanation ased on a single diension for est and worst referenes at the leel of otions, if one is willing to ake ertain assutions aout the aing fro attriute-leels to otions. These assutions ould inlude a reresentation at the otion leel that is a ultiliatie and equally-weighted funtion of the reresentation at the attriute leel. We use a odel-ased aroah to test this assution. Model-Based Aroahes to Testing Diensionality We now uild on the results aoe to test a seifi instane of a unified (onediensional) odel for est and worst hoies. If a single latent ariale generated the osered data in Exerients and 2, then a quantitatie odel that assues a single latent ehanis should aount for all asets of the data hoies, resonse ties, and araeter estiates onsistent with the data analysis. The artiular odel we used to test this hyothesis was the Linear Ballisti Auulator (LBA; Brown & Heathote, 2008). We ould hae used any of the other auulator-ased odels of onsuer hoie (e.g., Bhatia, in ress; Buseeyer & Townsend, 992; Otter, Alleny, & an Zandt, 2008; Roe, Buseeyer, & Townsend, 200; Usher & MClelland, 2004), ut we hose the LBA due to its outational siliity. In reious work we deonstrated that the LBA an e fit to est-worst hoies (Hawkins et al., in ress). When only hoies are odeled, and with ertain restritions on its araeters, the LBA is a rando utility odel, siilar to the ultinoial logit (Lue hoie) odel (Lue, 959; Marley & Flynn, in ress). Rando utility odels of est-worst hoie tyially assue that the utility, or referene strength, of an attriute-leel (or, soeties, an otion) that underlies est hoies also underlies worst hoies, with the releant worst alue utility equal to the negatie of the est utility (e.g., Marley & Pihlens, 202). In the following setion we roide a rief oeriew of the LBA odel in general, and the est-worst LBA in artiular. We then deonstrate that the latter tye of odel, whih assues a single reresentation for est and worst utilities, roides a good aount of hoie and resonse tie data. To foreshadow our results, the odeling led to three onlusions that onerge with the Bayesian and state-trae analyses of Exerients and 2: ) est utilities estiated fro est-only hoie data are highly orrelated with est

16 DIMENSIONALITY OF BEST AND WORST CHOICES 6 utilities estiated fro est-worst hoie data, and siilarly for worst utilities; 2) odels that allow est and worst utilities to differ in a ossily non-onotoni fashion neertheless estiate onotonially related est and worst utilities fro data; and 3) odels that assue an inerse relationshi etween est and worst utilities roide a good aount of data. In the odel fits we also illustrate two ethodologial oints: est-worst disrete hoie tasks roide greater onstraint on araeter estiates than traditional est-only disrete hoie tasks; and, siilarly, using oth hoies and resonse ties laes greater onstraint on araeter estiates than hoies, only. The Linear Ballisti Auulator Model The LBA shares the eidene auulation assution oon to ost odels of sile eretual hoies (e.g., Brown & Heathote, 2005; Ratliff, 978; Ratliff & Rouder, 998; Usher & MClelland, 200; an Zandt, Colonius, & Protor, 2000; for reiew see Lue, 986; Ratliff & Sith, 2004). The odel assues that a deision etween n otions is oosed of a rae etween n indeendent auulators that ollet eidene in faor of the aailale hoie alternaties aording to a drift rate, d, whih reresents the utility for eah otion. When the eidene in one auulator reahes a re-deterined riterion the resonse threshold, a resonse is triggered. The auulator that reahes threshold deterines the resonse, and the redited resonse tie is the tie taken for the auulator to reah threshold lus a fixed offset (non-deision tie, t 0 ), whih aounts for the tie inoled in stiulus enoding and otor roesses inoled in resonse rodution. The uer row of Figure 6 gies an exale of an LBA deision etween otions A, B, and C, reresented as indeendent auulators that rae against eah other. The ertial axes reresent the quantity of eidene, horizontal axes the assage of tie. At the eginning of a deision, the aount of eidene in eah auulator (the start oint ) aries indeendently aross auulators and randoly fro hoie-to-hoie, saled fro a unifor distriution etween zero and A (a araeter of the odel). Eidene auulates linearly, reresented as the arrows within eah auulator. The rate of eidene auulation is defined y the drift rate, whih is assued to ary randoly aross auulators and fro hoie-to-hoie aording to indeendent sales fro a Gaussian distriution: N(d, s), trunated to nonnegatie alues in our ileentation. Mean drift rate (d) reflets the attratieness (or utility) of an otion: a larger alue gies a faster rise to threshold and a ore likely hoie outoe, on aerage. Variaility in drift rates is assued to reflet deision-to-deision hanges in extraneous fators, suh as otiation or flutuations in attention. The general LBA fraework has roen suessful in aounting for the osered ariaility in the joint distriution of hoies and resonse ties aross a range of eretual hoie aradigs (e.g., Brown & Heathote, 2008; Forstann et al., 2008; Ho, Brown, & Serenes, 2009; Ludwig, Farrell, Ellis, & Gilhrist, 2009). We reiously exlored a nuer of ways to extend the LBA odel to aount for est-worst hoie tasks (Hawkins et al., in ress). In the referred extension, whih we alled the arallel est-worst LBA, we assued that est and worst hoies were the result of two arallel raes: a est rae and a worst rae, shown in the uer and lower rows of Figure 6. This odel aounted for the hoies and resonse ties fro a eretual hoie task, and the hoie data (when resonse ties were unaailale) fro two alied est-worst saling data sets. The arallel est-worst LBA ade a siilar assution aout

17 DIMENSIONALITY OF BEST AND WORST CHOICES 7 drift rates as the rando utility reresentation of the axdiff odel for est-worst hoies (see Marley & Louiere, 2005; Marley & Pihlens, 202): if d(x) is the drift rate for otion x in the rae to deide the est otion, then /d(x) is the drift rate for the sae otion in the rae to deide the worst otion. This inerse assution on drift rates ilies that those otions frequently hosen as est are also infrequently hosen as worst, and ie ersa. Our state-trae analyses of Exerients and 2 suort this assertion, ut exliitly fitting the odel to data roides a ore reise test of the sae hyothesis. If there is a non-onotoni relationshi etween est and worst hoies if seleting an otion as est is qualitatiely different to seleting an otion as worst then a odel whih enfores this inerse relationshi will qualitatiely isfit the data. Furtherore, we an fit odels that assue no relationshi etween est and worst drift rates (i.e., where all drift rates are free araeters) and use odel seletion tehniques to deterine whether the inerse relationshi etween est and worst drift rates roides the ost arsionious (and well-fitting) aount of the data. We fit the standard LBA to the est-only and worst-only onditions, and the estworst LBA to the est-worst onditions, of Exerients and 2. We first fit the odels to hoie roortions only, y integrating oer the distriution of redited resonse ties to otain arginal hoie roailities (for detail see Hawkins et al., in ress). We then fit the odels to the joint distriution of hoies and resonse ties to the data fro the est-only, worst-only, and est-worst onditions of Exerients and 2. Estiating Model Paraeters fro Data We fit the odels to indiidual artiiant data fro Exerients and 2 ia axiu-likelihood estiation using differential eolution otiization algoriths. In these odel fits we were riarily interested in estiating the drift rates; those rates easure the attratieness of different attriute-leels and, onsequently, of otions. 5 In Exerient we estiated nine drift rate araeters, one for eah retangle area. 6 In Exerient 2 there were too any ossile hone rofiles (243) to estiate a drift rate for eah, so we assued that the drift rate for a rofile was defined y the rodut of drift rate ontriutions fro its attriutes, onsistent with Marley and Pihlens (202) onstraints on their ultinoial logit odel araeters. The reiroal relationshi assued etween est and worst drift rates at the leel of attriutes, and the ultiliatie and equal-weighted reresentation for the drift rate for an otion in ters of the attriute-leel drift rates, leads to a reiroal relationshi etween est and worst drift rates at the otion leel. The alidity of this assution is therefore ealuated y how well the odel fits the data. Hene, in Exerient 2 we estiated 0 drift rates, orresonding to the 5 different attriute-leels, with one leel in eah attriute aritrarily onstrained suh that the rodut of the drift rates for the leels within eah attriute was equal to one (f. Marley & Pihlens, 202). When jointly fitting to hoies and resonse ties we set s = to fix a sale for 5 For hoie odels that an e written in the standard (additie) rando utility for for instane, the axdiff odel for est-worst hoie results to date suggest that the utility estiates for suh fors are the log of the drift rate estiates in the analogous LBA odel (Hawkins et al., in ress). 6 We did not estiate attriute-leel drift rates for retangle width and height in Exerient as we did for the attriutes studied in Exerient 2.

18 DIMENSIONALITY OF BEST AND WORST CHOICES 8 the eidene auulation roess, and estiated fro data the range in the start-oint (A), resonse threshold (), and non-deision tie (t 0 ). In the est-worst ondition we estiated a searate resonse threshold for the est and worst raes ( est, worst ). We also assued a fixed roaility (2%) of a unifor ontainant ixture roess (Ratliff & Tuerlinkx, 2002), interreted syhologially as a sall roaility of a failure of attention to the required judgent aross trials, whih also seres to stailize axiu likelihood araeter estiates. When fitting to hoie roortions only, the tie-related araeters of the odel are not identifiale, and were aritrarily fixed at s =, =, A = 0 and t 0 = 0 (for details see Hawkins et al., in ress). Thus, we estiated 9 (Exerient ) or 0 (Exerient 2) drift rates when fitting to hoies-only, and an additional three (est-only, worst-only) or four (est-worst) araeters when fitting the joint distriution of hoies and resonse ties. We ealuated goodness of fit using two ethods. We first oared osered and redited attriute-leel hoie roortions, searately for est and worst hoies. That is, for eah artiiant we alulated the roortion of ties that eah attriute-leel was hosen (that is, was in the hone that was hosen) relatie to the nuer of ties eah attriute-leel was aailale as a to-e-seleted otion, oth in data and odel reditions. In Exerient eah artiiant therefore roided 9 est or worst hoie roortions in the est-only and worst-only onditions, one for eah retangle area, and those in the estworst ondition roided 9 est and 9 worst hoie roortions. Siilarly, eah artiiant in Exerient 2 roided 5 est (and/or worst) attriute-leel hoie roortions. We alulated these alues fro indiidual artiiant data and fits to indiidual artiiant data, ut resent oth tyes of results aeraged oer artiiants. When fitting to hoies and resonse ties, we exained the orresondene etween osered and redited distriutions of resonse ties and hoie roortions. As with the hoie roortions, een though we fit the odels to indiidual artiiant data, to silify exosition we dislay the fit of the odel reditions aggregated oer artiiants. After the odels were fit to indiidual artiiants, we aggregated osered and redited resonse tie distriutions y alulating searately for eah ondition and artiiant the st, 5 th, 0 th, 5 th,..., 90 th, 95 th, 99 th erentiles of the resonse tie distriution, and then aeraging these indiidual artiiant quantiles to for an aggregate distriution of resonse ties. Quantile aeraging of this sort onseres distriution shae, under the assution that indiidual artiiant distriutions differ only in sale and loation (Gilhrist, 2000). We then onerted the aeraged quantiles to dislay distriutions in a histogra-like for. Model Fit to Data The odels roided a good fit to resonse tie and resonse hoie data fro the est-only, worst-only, and est-worst onditions of Exerients and 2 see Figure 7. The odel atured all qualitatie and ost quantitatie trends in the aggregate resonse tie distriutions: a sudden onset with a shar eak and a long ositiely skewed tail. Soe asets of the odel fit are not quantitatiely reise, suh as saller ariane in data than odel reditions for the est-only ondition of Exerient, and the est-worst

19 DIMENSIONALITY OF BEST AND WORST CHOICES 9 ondition of Exerient 2, ut all general trends were atured. 7 The fits to resonse tie distriutions illustrate two iortant oints. Firstly, est-worst hoie tasks do not result in unusual resonse ties. Seondly, this is one of the first deonstrations that a single quantitatie odel an aount for arkedly different resonse lateny data, ranging fro aroxiately one seond u to 30 seonds, fro sustantially different tasks eretual judgents of area and onsuer-like referenes for oile hones. We onsider this generality a strength of the LBA odels resented here. When fit to hoies and resonse ties, the odels roided a good aount of hoie roortions fro oth Exerients, aleit with soe underestiation of large hoie roortions see inset anels of Figure 7. In Exerient there was strong agreeent etween odel and data for the est-only ondition (R 2 =.95, root-ean-squared redition error 7.97%), worst-only ondition (R 2 =.98, 7.64%), and est-worst ondition (est R 2 =.95, 9.93%, worst R 2 =.96, 8.47%). The odels also roided a good aount of hoie data in Exerient 2 (est-only: R 2 =.88, 5.02%; worst-only: R 2 =.88, 5.00%; est-worst: est R 2 =.87, 5.49%, worst R 2 =.90, 4.8%). When the odels were fit to hoie roortions only, they also roided an exellent aount of the hoie data. 8 The fit to hoie data in Exerient 2 was slightly worse than in Exerient due to the olex, ulti-attriute oile hone stiuli and the ore sujetie nature of the deisions, whih rodue inherently noisier resonses than the sile retangles used in Exerient. Desite this, the odels roided a good aount of all resonse data. Paraeter Estiates Figure 8 dislays araeter estiates for the odel fits to hoies and resonse ties in the est-worst onditions. The uer anels of Figure 8 dislay the nine estiated drift rates for the retangular stiuli searated into the height and width diensions. As exeted, as eah diension inreases in size so does the distriution of drift rate estiates. This results in a ore likely est resonse for retangular stiuli with larger area, and onersely a ore likely worst resonse for retangles with saller area. Siilarly, in Exerient 2 the estiated drift rates followed the trends exeted fro the hoie data. For exale, as the rie of a hone inreased, the drift rate dereased. All of the other odel fits resulted in siilar araeter estiates. Regardless of whether one onsidered data fro the est-only, worst-only, or est-worst onditions, the utility onlusions drawn fro the attriute-leel drift rate estiates led to siilar onlusions; there were strong orrelations etween the edian araeter estiates fro eah oination of onditions aross oth exerients see Tale 2. That the est-worst odels roide a good fit to data suggests that the inerse assution on drift rates the drift rate assoiated with seleting the worst otion is the reiroal of the drift rate assoiated with seleting the est otion is reasonale. It also suggests that the ultiliatie and equal-weighted reresentation for otion-leel 7 A ore detailed fit ould e ade y setting the drift rate ariaility to one for est hoies, and estiating it for worst hoies. 8 Goodness of fit to hoie data when the odels were fit to hoie roortions, only: Exerient est-only, R 2 (RMSE:.7%); worst-only, R 2 (.48%); est-worst, est R 2 =.98 (3.70%); worst R 2 =.95 (6.07%); Exerient 2 est-only, R 2 =.98 (.7%); worst-only, R 2 =.97 (2.7%); est-worst, est R 2 =.95 (2.65%), worst R 2 =.96 (2.28%).

20 DIMENSIONALITY OF BEST AND WORST CHOICES 20 Tale 2: Correlation oeffiients (r) etween edian drift rate estiates fro the LBA odel fits to the est-only (B), worst-only (W), and est-worst (BW) onditions of Exerients and 2 (all s <.00; RT = resonse tie). Model fit B s. BW W s. BW B s. W Exerient Choies-only Choies & RT Exerient 2 Choies-only Choies & RT drift rates in ters of attriute-leel drift rates used in Exerient 2 is aetale, whih suorts an inerse relationshi etween est and worst referenes for otions. Howeer, we an ore diretly test the inerse assution on drift rates y allowing the odel to estiate different drift rate (utility) araeters for the est and worst raes. This allows, for exale, the ossiility that soe otions ight e frequently hosen as est and worst (as in Shafir, 993). This assution is not easily heked without resonse tie data, eause the hoie-only odels oe lose to saturation (i.e., nearly as any free araeters as data oints). Therefore, for oth Exerients and 2, we oared the fit of the est-worst LBA odel to oth hoies and resonse ties when ileenting the inerse assution (as reorted aoe) to the fit of a odel that had a searate set of drift rates for the est and worst raes, and hene 9 (Exerient ) or 0 (Exerient 2) additional free araeters in fitting data. The ore olex odel that allows searate utilities for est and worst hoies ust neessarily fit the data etter than the sile inerse odel whih it nests. To alane goodness of fit against odel olexity, we used the Bayesian Inforation Criterion (BIC; Shwarz, 978) and the Akaike Inforation Criterion (AIC; Akaike, 974). AIC is uh ore lenient than BIC in its treatent of olexity, so will tend to refer ore olex odels. For the eretual stiuli used in Exerient, the siler inerse odel was referred for 4 of 20 artiiants aording to BIC, ut only 2 of 20 artiiants aording to AIC. The ean differene in seletion riteria etween the inerse and free odels was equioal for BIC ( BIC M = 4.4, SD = 38.7, where negatie ean differene indiates suort for the inerse odel) ut not for AIC ( AIC M = 34.7, SD = 38.7). For the hone stiuli used in Exerient 2, the inerse odel was strongly referred for all 30 artiiants aording to BIC ( BIC M = 48., SD = 6.) ut the eidene was ore equioal for AIC, with the inerse odel referred for 7 of 30 artiiants ( AIC M = 3.7, SD = 6.). The AIC and BIC oarisons aoe are iased in faor of the olex odel in the sense that they test just one artiular ersion of the onstrained odel (a ersion where the drift rate for the worst hoie is exatly the reiroal of the drift rate for the est hoie). In a ore general sense, the drift rate estiates fro the free odel were highly onsistent with a onstrained exlanation that assues a single underlying soure for est and worst hoies. The edian est and worst drift rates in the free odel were strongly negatiely orrelated for oth Exerients and 2, r =.99, t(7) = 8.9, <.00 and

21 DIMENSIONALITY OF BEST AND WORST CHOICES 2 r =.9, t(3) = 7.8, <.00, resetiely, suggesting that est and worst drift rates roide siilar inforation. These results roide yet ore suort that est and worst hoies an e exlained with a single latent diension. Constraining Paraeter Estiates with Best-Worst Saling and Resonse Ties It is lausile that, relatie to traditional (est-only) disrete hoie tasks, the two resonses eliited in est-worst hoie tasks roide greater inforation (Vereulen, Goos, & Vanderoek, 200). One reason the two resonses ay roide greater onstraint is eause reiously ety ells in the design atrix are filled with useful inforation otions that are undesirale translating into ore reise araeter estiates in the estworst oared to est-only and worst-only onditions. We diretly tested this hyothesis. For eah fitting ode (hoies-only, hoies and resonse ties), ondition (est-only, worstonly, est-worst) and exerient we onduted searate reeated easures ANOVAs with the estiated drift rates for eah of the attriute-leels as the reeated easures fator. We used these ANOVAs to define the reision of araeter estiates as the aount of ariane exlained y the attriute-leels: R 2 = SSerror SS total. This aroah was ased on the assution that additional data (a seond resonse, and/or the resonse ties) will redue the roortion of unexlained ariane in the araeter estiates. When fit to hoie roortions only, drift rate estiates for the retangular stiuli in Exerient were ore strongly related to the attriute-leels in the est-worst ondition (R 2 =.72) oared to the est-only ondition (R 2 =.54), ut not the worst-only ondition (R 2 =.78). A siilar attern was osered for the oile hone data fro Exerient 2: the est-worst ondition (R 2 =.46) roided an iroeent oer the worst-only, ut not est-only, ondition (R 2 =.37 and R 2 =.46, resetiely). Siilarly to the addition of a seond hoie in est-worst saling, we exeted that fitting odels to hoies and resonse ties would roide further onstraint on araeter estiates. This is exatly what haened for all three onditions in Exerient adding resonse ties redued error in the araeter estiates oared to fitting hoie roortions, only (est-only R 2 =.73 s..54; worst-only R 2 =.83 s..78; est-worst R 2 =.8 s..72). For Exerient 2 data, fitting the odel to hoies and resonse ties inreased error in the araeter estiates oared to hoies, only, for the est-only ondition (R 2 =.40 s. R 2 =.46), led to a ild iroeent for the worst-only ondition (R 2 =.4 s. R 2 =.37), ut roided no hange relatie to the est-worst ondition (R 2 =.46). Our analysis suggests that otaining a seond different tye of resonse (worst or est), and/or olleting resonse tie inforation, generally roides greater onstraint on odel araeter estiates. It aears that there ight e an aroxiate uer liit on the roortion of ariane that relatiely sile odels suh as the LBAs fitted here an exlain in araeter estiates (e.g., aroxiately R 2.80 and R 2.45 in Exerients and 2, resetiely). Our results suggest that adoting either aroah (olleting lateny data or a seond resonse) an redue the roortion of unexlained ariane and lead to ore stale araeter estiates, whih erits ore reliale onlusions to e drawn.

22 DIMENSIONALITY OF BEST AND WORST CHOICES 22 General Disussion We exained seletion of the est and the worst otion through the lens of disrete hoie tasks, and est-worst saling in artiular, with the ai of inestigating whether the two odes of judgent an arise fro a single ognitie onstrut. Eidene fro two exerients one eretual and one onsuer analyzed with three statistial aroahes Bayesian analysis, state-trae analysis, and ognitie odeling onerged on a single thesis. In artiular, the delieration inoled in seleting the worst otion fro a set does not influene referenes for the est otion, and, equally, seleting whih otion is est does not influene referenes for the worst otion. Furtherore, est and worst hoies aear to reflet judgents that arise fro a single latent ariale. Our finding that atterns of est and worst hoies an e exlained y a single latent diension ay aear to e at odds with reious deonstrations that aeting and rejeting an e inonsistent (e.g., Shafir, 993; Tsetsos et al., 202); for exale, these authors reort that the sae otion an e oth aeted and rejeted under different task fraing aniulations. Howeer, in the Introdution, we hae already disussed the fat that est/worst and aet/rejet do not gie the sae inforation. A further key differene etween these reious studies and the researh reorted here is that we did not aniulate the ariaility in the attriutes orising the hoie otions. For exale, in Tsetsos et al. s (202) task, streas of nuers were generated fro two Gaussian distriutions, one with a higher ariane than the other, and resondents were asked to ealuate the ean alue of eah strea. When deisions were searated oer tie, resondents tended to oth rejet and aet the high ariane otion when gien different task goals. This roedure has two iortant differenes fro our tasks, and fro any standard hoie exerients. Firstly, in our tasks deisions, when est and worst hoies were ade, they were ade to eah hoie set, and therefore not as widely searated oer tie as in Shafir (993) and Tsetsos et al. (202). Seondly, our stiulus sets were not aniulated to ontain high and low ariane otions (equialently, enrihed or ioerished otions, Shafir, 993). Furtherore, these studies used dissoiation logi to suort the thesis that aeting is inonsistent with rejeting. Future researh is required to deterine whether aniulations suh as Tsetsos et al. s rodue inonsisteny etween est and worst hoies when analyzed with a ethodology that addresses the issue of onfounding y sale-deendent interations. Here, we diretly addressed this issue with state-trae analysis, whih assesses the ordinal relationshis etween exeriental fators to oeroe the effets of range-restrited resonse ariales. With this analysis we deonstrated onsisteny etween est and worst hoies. Another ossile exlanation for the disreany etween Shafir s (993) and Tsetsos et al. s (202) and our results is the diergent goals of researhers using roess-ased ersus easureent-ased odels. For exale, Buseeyer and Rieska (203) desrie how researhers using easureent odels ai to aly statistial odels to large sales of (disrete) hoie data aggregated oer eole, to effiiently estiate araeters whih an e iortant to eonoists while roess-foused researhers ai to understand the ognitions underlying hoie y indiiduals whih an e iortant to syhologists. This fous has led syhologists to onstrut artiular exeriental situations that yield interesting ontext effets, whih in turn reeal sutle ognitie iases. In ontrast, eonoists

23 DIMENSIONALITY OF BEST AND WORST CHOICES 23 tend to fous on easureent using designs that aoid suh effets. In this aer we aied to ring together the oasionally oosing goals of hoie odelers and syhologists, y alying a ognitie odel tyial of the syhologial literature to an exeriental task tyial of the disrete hoie literature. The LBA odel alied in this aer does not naturally aount for ontext effets, though it an e extended to do so (Truelood, Brown, & Heathote, in ress). It is likely that the est-worst LBA ileented here an siilarly e extended to aount for known ontext effets in est-worst hoie. For exale, Shafir s (993) finding that the su of the roortion of ties that an otion is aeted under one instrution and rejeted under a seond instrution an e greater than suggests aeting and rejeting are not onsistent; we should also reeer that Shafir (993) aniulated the two instrution sets etween sujets. Howeer, fraing (ontext) effets suh as Shafir s (993) ight e fit y a weighted ultiattriute odel where the weights hange aross tasks, ut the underlying utilities do not. Thus, if resondents are guided to use different deision rules for aeting and rejeting, we ay find different results for the two tasks. Howeer we hae found that in unguided tasks, with standard est, resetiely worst, instrutions, suh effets tend not to arise. In our results, the roortion of ties that eah stiulus otion was seleted as est did not differ aross the est-only and est-worst hoie onditions, for either eretual or onsuer stiuli (Exerients and 2). This result is iortant sine the est resonse is usually of greater interest to exerienters, artiularly in alied doains. That is, for a arketer it is likely ore rofitale to know whih rodut onsuers ight urhase, rather than the (otentially any) roduts they likely will not urhase. Our results suggest that data on est, or worst, hoies an e used interhangealy to yield siilar onlusions aout referene strengths, though the est-worst roedure likely rodues ore reliale araeter estiates for odel-ased inferene. Aknowledgents This researh has een suorted y Natural Siene and Engineering Researh Counil Disoery Grant to the Uniersity of Vitoria for Marley, and y Australian Researh Counil grants FT and DP to the Uniersity of Newastle for Brown and Heathote. The work was arried out, in art, whilst Marley was a Distinguished Professor (art-tie) in the Centre for the Study of Choie, Uniersity of Tehnology, Sydney.

24 DIMENSIONALITY OF BEST AND WORST CHOICES 24 Referenes Akaike, H. (974). A new look at the statistial odel identifiation. IEEE Transations on Autoati Control, 9, Baer, D. (979). State-trae analysis: A ethod of testing sile theories of ausation. Journal of Matheatial Psyhology, 9, Bhatia, S. (in ress). Assoiations and the auulation of referene. Psyhologial Reiew. Brown, S., & Heathote, A. (2005). A allisti odel of hoie resonse tie. Psyhologial Reiew, 2, Brown, S., & Heathote, A. J. (2008). The silest olete odel of hoie reation tie: Linear allisti auulation. Cognitie Psyhology, 57, Buseeyer, J. R., & Rieska, J. (203). Psyhologial researh and theories on referential hoie. In S. Hess & A. Daly (Eds.), Handook of Choie Modelling. Edward Elgar Pulishing. Buseeyer, J. R., & Townsend, J. T. (992). Fundaental deriations fro deision field theory. Matheatial Soial Sienes, 23, Busey, T. A., Tunniliff, J., Loftus, G. R., & Loftus, E. F. (2000). Aounts of the onfidene auray relation in reognition eory. Psyhonoi Bulletin & Reiew, 7, Cheng, P. Y., & Chiou, W. B. (200). Rejetion or seletion: Influene of fraing in inestent deisions. Psyhologial Reorts, 06, Dunn, J. C., & Kirsner, K. (988). Disoering funtionally indeendent ental roesses: The rinile of reersed assoiation. Psyhologial Reiew, 95, 9 0. Dunn, J. C., Newell, B. R., & Kalish, M. L. (202). The effet of feedak delay and feedak tye on eretual ategory learning: The liits of ultile systes. Journal of Exeriental Psyhology: Learning, Meory, and Cognition, 38, Finn, A., & Louiere, J. J. (992). Deterining the aroriate resonse to eidene of uli onern: The ase of food safety. Journal of Puli Poliy and Marketing,, Forstann, B. U., Dutilh, G., Brown, S., Neuann, J., on Craon, D. Y., Ridderinkhof, K. R., et al. (2008). Striatu and re SMA failitate deision aking under tie ressure. Proeedings of the National Aadey of Siene, 05, Ganzah, Y. (995). Attriute satter and deision outoe: Judgent ersus hoie. Organizational Behaior and Huan Deision Proesses, 62, Ganzah, Y., & Shul, Y. (995). The influene of quantity of inforation and goal fraing on deision. Ata Psyhologia, 89, Gilhrist, W. (2000). Statistial odelling with quantile funtions. London: Chaan & Hall/CRC. Hawkins, G. E., Marley, A. A. J., Heathote, A., Flynn, T. N., Louiere, J. J., & Brown, S. D. (in ress). Integrating ognitie roess and desritie odels of attitudes and referenes. Cognitie Siene. Ho, T., Brown, S., & Serenes, J. (2009). Doain general ehaniss of eretual deision aking in huan ortex. Journal of Neurosiene, 29, Huer, V. L., Neale, M. A., & Northraft, G. B. (987). Deision ias and ersonnel seletion strategies. Organizational Behaior and Huan Deision Proesses, 40, Juliano, L., & Wilox, K. (20). Choie, rejetion, and elaoration on referene-inonsistent alternaties. Journal of Consuer Researh, 38, Lein, I. P., Jaser, J. D., & Fores, W. S. (998). Choosing ersus rejeting otions at different stages of deision aking. Journal of Behaioral Deision Making,, Lein, I. P., Prosansky, C. M., Heller, D., & Brunik, B. M. (200). Presreening of hoie otions in ositie and negatie deision aking tasks. Journal of Behaioral Deision Making, 4, Loftus, G. R. (978). On interretation of interations. Meory & Cognition, 6, Loftus, G. R., & Masson, M. E. J. (994). Using onfidene interals in within sujet designs. Psyhonoi Bulletin & Reiew,,

25 DIMENSIONALITY OF BEST AND WORST CHOICES 25 Loftus, G. R., Oerg, M. A., & Dillon, A. M. (2004). Linear theory, diensional theory, and the fae inersion effet. Psyhologial Reiew,, Lue, R. D. (959). Indiidual hoie ehaior: A theoretial analysis. New York: Wiley. Lue, R. D. (986). Resonse ties. New York: Oxford Uniersity Press. Ludwig, C. J. H., Farrell, S., Ellis, L. A., & Gilhrist, I. D. (2009). The ehanis underlying inhiition of saadi return. Cognitie Psyhology, 59, Marley, A. A. J., & Flynn, T. N. (in ress). Best and worst saling: Theory and aliation. In J. D. Wright (Ed.), International enyloedia of the soial and ehaioral sienes (2nd ed.). Elseier. Marley, A. A. J., & Louiere, J. J. (2005). Soe roailisti odels of est, worst, and est worst hoies. Journal of Matheatial Psyhology, 49, Marley, A. A. J., & Pihlens, D. (202). Models of est worst hoie and ranking aong ultiattriute otions (rofiles). Journal of Matheatial Psyhology, 56, Morey, R. D. (2008). Confidene interals fro noralized data: A orretion to Cousineau (2005). Tutorials in Quantitatie Methods for Psyhology, 4, Morey, R. D., & Rouder, J. N. (203). BayesFator: Coutation of Bayes fators for sile designs [Couter software anual]. Aailale fro htt://cran.r-rojet.org/akage=bayesfator (R akage ersion 0.9.4) Newell, B. R., & Dunn, J. C. (2008). Diensions in data: Testing syhologial odels using state-trae analysis. Trends in Cognitie Sienes, 2, Newell, B. R., Dunn, J. C., & Kalish, M. (200). The diensionality of eretual ategory learning: A state-trae analysis. Meory & Cognition, 38, Otter, T., Alleny, G. M., & an Zandt, T. (2008). An integrated odel of disrete hoie and resonse tie. Journal of Marketing Researh, 45, Prine, M., Brown, S., & Heathote, A. (202). The design and analysis of state-trae exerients. Psyhologial Methods, 7, R Deeloent Core Tea. (203). R: A language and enironent for statistial outing [Couter software anual]. Vienna, Austria. Aailale fro htt:// (ISBN ) Ratliff, R. (978). A theory of eory retrieal. Psyhologial Reiew, 85, Ratliff, R., & Rouder, J. N. (998). Modeling resonse ties for two hoie deisions. Psyhologial Siene, 9, Ratliff, R., & Sith, P. L. (2004). A oarison of sequential saling odels for two hoie reation tie. Psyhologial Reiew,, Ratliff, R., & Tuerlinkx, F. (2002). Estiating araeters of the diffusion odel: Aroahes to dealing with ontainant reation ties and araeter ariaility. Psyhonoi Bulletin & Reiew, 9, Roe, R. M., Buseeyer, J. R., & Townsend, J. T. (200). Multi alternatie deision field theory: A dynai artifiial neural network odel of deision aking. Psyhologial Reiew, 08, Saille, D. J. (2003). Basi statistis and the inonsisteny of ultile oarison roedures. Canadian Journal of Exeriental Psyhology, 57, Shwarz, G. (978). Estiating the diension of a odel. Annals of Statistis, 6, Shafir, E. (993). Choosing ersus rejeting: Why soe otions are oth etter and worse than others. Meory & Cognition, 2, Truelood, J. S., Brown, S. D., & Heathote, A. (in ress). The ulti-attriute linear allisti auulator odel of ontext effets in ulti alternatie hoie. Psyhologial Reiew. Truelood, J. S., Brown, S. D., Heathote, A., & Buseeyer, J. R. (203). Not just for onsuers: Context effets are fundaental to deision aking. Psyhologial Siene, 24, Tsetsos, K., Chater, N., & Usher, M. (202). Saliene drien alue integration exlains deision

26 DIMENSIONALITY OF BEST AND WORST CHOICES 26 iases and referene reersal. Proeedings of the National Aadey of Siene, 09, Tersky, A. (972). Eliination y asets: A theory of hoie. Psyhologial Reiew, 79, Usher, M., & MClelland, J. L. (200). On the tie ourse of eretual hoie: The leaky oeting auulator odel. Psyhologial Reiew, 08, Usher, M., & MClelland, J. L. (2004). Loss aersion and inhiition in dynaial odels of ultialternatie hoie. Psyhologial Reiew,, an Buiten, M., & Keren, G. (2009). Seakers hoie of frae in inary hoie: Effets of reoendation ode and otion attratieness. Judgent and Deision Making, 4, an Zandt, T., Colonius, H., & Protor, R. W. (2000). A oarison of two resonse tie odels alied to eretual athing. Psyhonoi Bulletin & Reiew, 7, Vereulen, B., Goos, P., & Vanderoek, M. (200). Otaining ore inforation fro onjoint exerients y est worst hoies. Coutational Statistis and Data Analysis, 54, Wagenakers, E.-J., Kryotos, A.-M., Criss, A. H., & Ierson, G. (202). On the interretation of reoale interations: A surey of the field 33 years after Loftus. Meory & Cognition, 40,

27 DIMENSIONALITY OF BEST AND WORST CHOICES 27 Prie () Best Choie Proortions (Best Worst Condition) Caera () Best Choie Proortions (Best Only Condition) Video () Meory () Worst Choie Proortions (Worst Only Condition) Battery () Figure 5. State-trae lots for the oile hone judgents in Exerient 2. The uer right anels lot attriute-leel est hoie roortions etween the est-only (y-axes) and est-worst (x-axes) onditions. The lower left anels lot attriute-leel est hoie roortions for the estonly ondition (y-axes) against attriute-leel worst hoie roortions for the worst-only ondition (x-axes). Eah anel dislays the o-ariation of attriute-leel ean hoie roortions etween the leels of two attriutes, with the figure deiting all airwise oarisons of attriutes. Ellises reresent etween-sujets least signifiant differenes. Monotoni ures added to aid isualization only and do not reresent est-fitting onotoni funtions to data. Axis saling oitted for larity. Note that axes are not equal in sale (see Figure 4).

28 DIMENSIONALITY OF BEST AND WORST CHOICES 28 Parallel Best-Worst LBA Best Rae Start Point Resonse Threshold Drift Rate Best resonse (fastest) Drift rates ary randoly fro trial to trial (noral distriution) Resonse A Resonse B Resonse C Worst Rae Deision Tie Resonse A Resonse B Worst resonse (fastest) Resonse C Start oints ary randoly fro trial to trial (unifor distriution) NOISE PROCESSES: Start Point is a i ~ Unifor(0,A) Drift Rate is d i ~ Noral (d,s) Figure 6. Illustratie exale of the deision roesses of the arallel est-worst linear allisti auulator odel. See ain text for full details.