Social Policy Targeting and Binary Information Transfer between Surveys

Transcription

1 Dsusson Paper No Marh 18, Soal Poly Targetng and Bnary Informaton Transfer between Surveys Danel Gottleb and Leond Kushnr Ben-Guron Unversty; formerly at the Bank of Israel Please te the orrespondng journal artle: Abstrat In ths paper we develop a methodology for dentfyng a populaton group surveyed latently n the (target) survey relevant for further proessng, for example poverty alulatons, but surveyed expltly n another (soure) survey, not sutable for suh proessng. Identfaton s aheved by transferrng the bnary nformaton from the soure survey to the target survey by means of a logst regresson determnng group afflaton n the soure survey by use of varables avalable also n the target survey. In the proposed methodology we mprove on ommon mathng proedures by optmzng the ut-value of the probablty whh assgns group afflaton n the target survey. Ths ontrasts wth the ommonly used "Hosmer-Lemeshov" ut-values for bnary ategorzaton, whh equates between the senstvty and spefty urves. Instead we mprove group dentfaton by mnmzng the sum of total errors as a perent of total true outomes. The Jewsh ultra-orthodox populaton n Israel serves as a ase study. Ths dosynrat ommunty, ommtted to the observane of the Bble s only latently observed n the surveys typally used for poverty alulaton. It s expltly aptured n the soal survey, whh s not sutable for poverty measurement. Ths proedure s useful for ex-post enhanement of survey data n general. JEL: C15, D63, I38, Z12 Keywords: Group dentfaton; bnary varables; optmal utoff value; poverty; targetng Correspondene: Danel Gottleb, Natonal Insurane Insttute and Eonoms Department of Ben- Guron Unversty, emal: danelgt@no.gov.l The authors bear sole responsblty for any vews or mstakes presented here. They thank the Central Bureau of Statsts, Israel for the support n preparng the data base. Author(s) Lensed under a Creatve Commons Lense - Attrbuton-NonCommeral 2.0 Germany

2 1. Introduton The Hared (Jewsh ultra-orthodox) populaton n Israel s an dosynrat ommunty, ommtted to the observane of the Bble and ts ommandments, as nterpreted by ts setaran relgous leaders. Hared poverty ndene s exeptonally hgh at 67.5%, wth a share of 20% of all Israel poor whle ts share n the total populaton s only half that sze. Its major auses are a very hgh Hared fertlty (a populaton growth of 6% p.a.), redung both household nome per apta and the mother's earnng apaty; ts eduaton system s largely ndependent from the natonal shool system and neglets (partularly among boys) materally mportant subjets for the buldup of future earnng apaty suh as Mathemats, Englsh and dgtal sklls; a low labor-fore partpaton of Hared men, due to prolonged learnng n relgous semnars (Yeshva), often deep nto the prme workng age. A further ause for the sharp nrease n short-term poverty has been the reent large uts n hld beneft payments. The share of Hared hldren up to age 4 s nearly 3 tmes hgher than n the rest of the Jewsh soety. Ths, together wth the empral regularty of a negatve relatonshp between poverty and age mples an upward-drft for Hared and overall Israel poverty over tme. Hared Poverty, as measured by the dstrbuton-senstve Sen-poverty ndex, nearly doubled over the last 3 years after a prevous sgnfant mprovement. Ths deteroraton stands n ontrast to developments n the rest of the Israel-Jewsh soety, whose poverty ntensty nreased only slghtly over the last ouple of years. In 2004 Hared male labor fore partpaton of 37% hardly exeeded one half that of the other Jewsh male populaton, manly due to the Hared hgh enrollment n relgous semnars (Yeshva) durng ther prme workng age. Despte ther muh hgher fertlty the women funton as the famly's man provders, wth a partpaton rate of 48%, ompared to 58% of non-hared women. Prelmnary and stll statstally nsgnfant empral evdene ponts to a reent nrease n Hared labor-market nvolvement, both among men and women, probably related to nreased eonom hardshp, maybe due to the drast ut n hld allowanes from 2002 to Empral evdene shows job tranng to affet labor fore entry postvely, partularly among Hared men, though at low wages. These shemes proved suessful tools when oneved wth a hgh senstvty towards the partular ultural needs of the Hared soety. Proper dentfaton of the poor s essental for soal poly targetng. When the poor belong to a spef ultural group wth exeptonally hgh poverty ndene, the bas determnants of ther poverty are typally entered on famly sze, eduatonal defenes and labor 2

3 market behavor. However, these haratersts mght have more deep-seated ultural roots, refleted n olletve preferenes onernng fertlty and gender-related dfferenes n eduaton and labor-fore partpaton. Suh underlyng ultural determnants whh are of a qualtatve rather than quanttatve nature should be nluded expltly n the poverty analyss. Suh nformaton s usually lakng n standard nome or onsumpton surveys typally used for poverty alulatons. Suh nformaton s to be found n speal soal surveys, based on the same populaton, but not sutable for standard poverty alulatons. The queston arses whether an effent proedure an be devsed by whh suh nformaton ould be transferred ex-post from the orgnal survey, the soure-survey, to the relevant surveys for poverty alulaton,.e. the target-surveys. Suh a proedure may be a useful tool for ex-post enhanement of the nformaton ontent of survey data n general. The major soal and eonom surveys of an eonomy fous typally on dfferent aspets of the same populaton. Whle some of the questons reur n more than one survey, other nformaton s unque to a spef survey. Sne the gatherng of nformaton s not ostless and some of the survey-spef nformaton mght be useful to researhers of another survey or to poly makers, we suggest an effent method for optmal bnary nformaton transfer (BIT) from one survey (the "soure" survey) to another (the "target" survey). The optmal method for transferrng the nformaton depends rually on three aspets of the proess: (1) an overlap of the set of varables that ontan explanatory power of the varable to be transferred (thus ensurng a reasonable goodness of ft of the Reever Operatng Charaterst-urve, heneforth ROC-urve), (2) the rule for determnng the utoff value,.e. the value by whh the logst probablty foreast s translated bak nto a bnary varable and (3) a qualty test of the proedure. Our qualty test, whle performed n the soure survey, stll provdes a lue to the qualty of the synthet nformaton n the target data set. Suh enhanement of soo-eonom data by an ex-post nformaton transfer s partularly useful when addtonal data olleton by a survey s ether too expensve or mpossble. Our method for hoosng the utoff value of the foreasted probablty s shown to mprove on that suggested by Hosmer and Lemeshow, BIT has several possble applatons. It an be useful n the targetng of poles to spef populaton groups, whh s one of the purposes of poverty mappng. 2 The present results 2 In reent years poverty mappng has beome an mportant tool n mprovng targetng. Ths tehnque utlzes nformaton from surveys, amenable to poverty alulatons, but too small for effent targetng of the poor, by transferrng nformaton to large sale data bases suh as ensus data, whh provde less detaled nformaton, but on a larger share of the populaton. Suh a transfer s arred out by use of eonometr tools. The purpose s to enable the alulaton of poly varables, for example bnary nformaton on poverty, for small geograph 3

4 mght also be used n medal researh and other researh employng logst regresson and utoff values. 3 We then llustrate the applaton of the method to the measurement of poverty n a spef group, known for ts hgh poverty ndene the Israel Jewsh Ultra-orthodox ("Hared") populaton. Due to the lak of nformaton on relgous afflaton n the surveys typally used for poverty alulatons (the nome- and expendture surveys), and the lak of suffently detaled nome and onsumpton data n the survey that does provde nformaton on relgous afflaton there arses a need for the transfer of nformaton on Hared membershp from the Soal Survey to the Inome and Expendture surveys. The paper s organzed as followng: In hapter 2 the model of BIT s presented. Chapter 3 desrbes the proess of BIT n more detal. In hapter 4 we report on a ase study of BIT appled to the Israel Hared populaton for the purpose of poverty alulatons. 4 Conludng remarks omplete the paper. 2. The Model Assume a samplng of two Household surveys, one whh we all the Soure-survey (S), onsstng of n S = 1 S households, and another survey sampled on the same populaton 5, whh we all the Target survey (T), n T = 1 T. Let there be a dhotom bnary group varable, say of group H, wth a value of 1 for suess and 0 for falure. We denote the household's probablty of event H = 1 ourrng, as π and ts estmate as ˆ π. The estmate s ondtonal, based on vetor x of explanatory varables, ˆP ( H =1 x') = πˆ (x), where vetor x ' = ( x1, x2,... x k ). s a bnary estmate of H for ndvdual n the respetve sample of the soure (S) or the target survey (T). Obvously, the suggested proedure requres vetor x' to appear n both S and T. The logst probablty funton for event H=1 s gven by Hˆ ST g ( x) e π ( x) =. (1) g ( x) 1 + e areas. See for example Hentshel, J., Lanjouw, J.O., Lanjouw, P. and Pogg, J. (2000), Bgman and Srnvasan (2002) or Small Area Estmaton at 3 See for example P.M. Dodek and B. R. Wggs, 1998, Hadjostas Petros and George C. Hadjnola, 2001, G. Shares et al., 2003, Shutter E.M.J. et al., 1998 and Stegeman, J.A. et al., A detaled analyss of poverty n the Israel Hared populaton, based on an earler verson of the present model s gven n Gottleb, Sne the households are hosen by spef mehanal proesses the hanes that the same household wll appear n more than one survey s neglgble. Of ourse f t does, and the researher knows that nformaton, then the nformaton transfer beomes trval. 4

5 The logt equaton nludes ontnuous (k=1 K) and ategoral varables (D jl, j=1 J), suh as smple dummy varables or dummy varables wth more detaled odng levels (l=1 L-1): L j 1 g( x) = β0 + β1x β jldjl + βkx. (2) K l= 1 3. The BIT proess Step 1: Searh for an Effent Logst Regresson n the Soure Survey The qualty of BIT depends rually on the explanatory power (not neessarly n a ausal sense) of equaton (2) of group membershp probablty n the Soure survey ( ˆ π S ). The better the explanatory power, as reported n the regresson's log-lkelhood rato, Wald test, the z- values and addtonal statstal parameters, the better s the hane for a suessful BIT of household 's group membershp. Step 2: A Foreast of Group Membershp, Usng a 'Contnuous' Cutoff Value ( ˆ π S ) We hoose any utoff pont 0 ˆ π S 1 n the soure survey, above whh the foreast of household 's group membershp ( Hˆ S ) s ether 1 or 0. We repeat ths proedure, overng the whole range of 0 π 1. Consequently, H = ˆ S ˆ S Ĥ ( ˆ π S ) for =1 S. For eah utoff value we then organze the bnary outomes of Hˆ ˆ π nto 4 mutually exlusve ategores: True Postve Outomes: TP( ˆ π S ) for all Ĥ = H = 1, True Negatve Outome: TN( ˆ π S ) for all Ĥ = H = 0, False Postve Outome: FP( ˆ π S ) for all Ĥ = 1 and H = 0, False Negatve Outome: FN( ˆ π S ) for all Ĥ = 0 and H = 1, These steps are repeated for a near-ontnuous number of utoff values. Step 3: Assessment of the Foreast Qualty n the Soure Survey The error rate or foreast qualty an only be estmated n the soure survey sne the target survey nludes only the set of explanatory varables and not the dependent varable. We haraterze the foreast qualty usng the ROC urve as a measure. Step 4: Searhng for the Optmal Probablty Cutoff Value ( π ) We hoose the optmal utoff value by usng the outomes of the prevous step,.e. the utoff value that mnmzes the sum of total squared errors FP and FN. Note that Hosmer and Lemeshow (heneforth HL) suggest that the optmal utoff value s at the level ˆ S,* π for whh ˆ S,* 5

6 senstvty equals spefty. In the followng we show that n the present ase our hoe yelds a sgnfant mprovement on the HL hoe. Step 5: The BIT - Calulaton of the Foreast Hˆ T n the Target-Survey After havng asertaned that we have elted the best possble foreast we move to the target survey. As mentoned before there s no way of testng the qualty of BIT, exept by new data olleton. We alulate ˆ T H by use of the regresson equaton and the optmal utoff value as estmated n the soure-survey. 4. A BIT Case Study: Poverty among the Jewsh Ultra-Orthodox n Israel The Israel Ultra-Orthodox Jewsh soety, also alled Hared soety, 6 has long been known to have an exeptonally hgh poverty ndene. However, sne there s no ndaton of Hared afflaton n the surveys used for estmatng poverty, avalable poverty studes and n partular the offal ones do not report separate poverty estmates for ths populaton group. Some studes have attempted to estmate poverty n ths populaton group on a natonal level and we shall dsuss them below. Due to ts dosynrat ultural features and a lak of ther explt nluson n survey questons the Hared soety s an nterestng example for applyng the BIT proess. Ther heterogeneous labor market behavor leads to extreme poverty stuatons of many Hared households. Consequently, there arses a need for statstal enhanement onernng Hared group membershp n the major surveys used for ratonal poly formulaton and mplementaton. In order to model a logst group membershp probablty of the Hared (the frst step of the BIT proess) we brefly haraterze them here. 4.1 The Israel Hared Soety Roots and Charatersts The Israel Hared soety s fragmented nto several subgroups, eah emphaszng dfferent aspets of Judasm and obeyng ts own sprtual leaders. For smplty we onentrate on three man fatons: The Hassd, the Lta' and the Sephard 7 groups. They all share strt observane of the Torah and the Jewsh ommandments and a hgh degree of omplane to ther sprtual leaders' desons onernng a wde range of publ and Famly ssues. The 6 "Hared" s the Hebrew name of the Ultra-Orthodox soety. It has the meanng of a person who "trembles n awe of God". It nludes dstnt groups, ommon n ther unequvoal ommtment to the study and observane of the Torah and ts ommandments, as nterpreted by ther relgous leaders. See also Fredman, Sephard orgnally ndated the Judeo-Spansh orgn. In the Israel ontext t s sometmes used n a wder ontext to ndate also other Jews orgnatng from North Afra or the Mddle East. 6

7 leadershp mantans a strong sense of herarhy and ssues and looks over detaled rules for ndvdual and famly behavor through ts organs. When n the early 20th entury seular Jewsh natonalsm emerged as a rapdly growng alternatve to the relgous way of lfe, the Hared rejeted ts ant-relgous harater strongly. Sne a hstoral ompromse n 1948 between Davd Ben-Guron (then Prme Mnster) and Hazon Ish 8 (then Leader of the Hared soety) male relgous sholars, whose man oupaton s Torah study, Hared women and men n draftng age are by and large exempted from servng n the Israel army. Over the years the number of exempted men from the army grew rapdly (8 to 9% p.a., reahng more than 30,000 n the early 21 st mllennum from about 400 n In response to ourt appeals and a general publ dsontent over that exempton from army serve and to a mountng soal problem of young drop-outs from the Hared relgous semnars, an offal ommsson, apponted n 1999, proposed a hange n government poly wth the purpose of redung the number of exemptons from the army and of mprovng Hared men's labor fore partpaton Estmates of Hared Populaton Sze Several attempts to estmate the sze of the Hared populaton were based on the queston about the "last shool vsted" n the household surveys of the ICBS. Ths eduaton-based approah was poneered by Berman and Klnov, 1997 and also Dahan, It was elaborated n Berman, 2000, and has sne then been used for analyzng Hared poverty and labor market behavor. 10 Other populaton estmates were based on eleton results due to typally monolth Hared votng-patterns The Eduaton-based approah Aordng to ths approah a household s assumed to be Hared f at least one of ts male members ndates a Yeshva (a relgous semnary) 12 as the last shool attended. Berman (2000) foreasted Hared populaton to reah 280,000 n 1995 and 510,000 people by 2010, based on expeted fertlty and death rates. Suh foreasts are bound to produe unsatsfatory results for a number of reasons: Yeshva studes do not onsttute a neessary ondton for 8 Rabb Abraham Yshayahu Kareltz, The ompromse nluded also an exempton of Hared grls. In the years 1951 and 1952 the argument over draftng Hared women developed nto a government rss, ausng the Hared party Agudat Israel to leave the government. 9 See the report of the Tal Commsson, See for example Flug and (Kalner) Kasr, 2003, Gottleb and (Kalner) Kasr, 2004 and Gottleb and Manor, See Degan and Degan, 2000 (heneforth DD), and more reently Gurovh and Cohen, 2004 (heneforth GC). 12 These semnares are not to be onfounded wth relgous Hgh shools (Yeshva Thont), whh ombne relgous studes wth a hgh shool urrulum. The latter are typally frequented by orthodox rather than ultraorthodox Judasm. Orthodox Jews, dstntly from the Ultra-orthodox are fully ntegrated n the Israel soety, ts labor market as well as n the army. 7

8 Hared belef. Indeed, Yeshva attendane among Hassd Jews,a large group wthn Hared soety, s beleved to be lower than n the Lta' and Sephard Hared groups The Eletons-based Approah Ths approah was hosen by GC, based on the 2003 eletons 13 and a geograph dentfaton of loaltes wth a hgh perentage of voters for the two poltal partes of Hared orentaton out of the 13 party lsts represented n the parlament: Unted Torah Judasm (UTJ, or n Hebrew "Yehadut HaTorah") and "Shas" 14. Whle the voters for UTJ are supposedly manly Hared, many "Shas" supporters are less relgous but rather tradtonal or ethnally orented voters. In order to dentfy ths subgroup, GC nluded Shas supporters among the Hared only f they lved n the vnty of areas wth a hgh perent of UTJ support, assumng that the Hared lke to lve whtn eah other's proxmty. GC onluded that only 1/3 of the Shas voters are Hared. The populaton estmate s alulated as followng: j H Pop = ( voters / pj)/(1-x j ) where = number of voters for eah party, j = UTJ party/shas party, where p j = eletonpartpaton rate of the j th party. x j = perent of populaton under votng age of the j th party supporters. In areas wth a hgh rate of UTJ voters, the researhers report a hgh partpaton rate ompared to other areas. In areas wth 90% and more UTJ votes the general partpaton rate was 94%. In areas wth 80% and more UTJ votes, the general partpaton rate was 85%. The study assumes a sgnfantly hgher Hared eleton partpaton rate than that of the general publ. 15 Based on fertlty rates derved from the Soal survey for Hared women of Ashkenas 16 bakground, GC used a fertlty of 7.5 brths per woman yeldng an estmate of the share of people below the votng age (based on a model of stable populatons) of 56% of the populaton. The total populaton of atual and potental UTJ voters s estmated to be 361,000. The Sephard Hared estmate amounted to 204,000 and the total Hared populaton was estmated at 565,000 by the end of The Estmate based on the Soal Surveys of the ICBS The frst Soal Survey wth a sample sze of some 10,000 persons aged 20 or more and ther household was publshed n The estmate of Hared afflaton s based on queston Nr. 13 An earler study by Degan and Degan, 2000, was based on the 1996 eletons. 14 The "Shas" party of Torah-observant Sephards was founded n It has many non-orthodox supporters. 15 The general partpaton rate n the 2003 eletons was 67.8%. When adjusted for the very low Arab eleton partpaton rate and for the Israels who were absent durng the eletons, the general partpaton rate s somewhat hgher but stll lower than the Hared partpaton rate. 16 In the present ontext ths ndates a European (nludng Eastern European and Russan) and Anglo-Saxon bakground. 8

9 26 of the questonnare 17. In order to estmate the populaton sze nludng hldren, the weghts need to be adjusted to aount for the fat that n a household there may be more than one person aged 20 or more. We alulate populaton sze by use of the followng formula 18 : nn HPop = nn+ under 20 over 20 where H = Hared populaton, = people delarng themselves as Hared and nn = Pop populaton weght for eah respondent. Aordng to the Soal Survey they were thousand by end of Under20 = number of people aged under 20 n the th (Hared) household and over20 = number of addtonal people (to those questoned) aged 20 or more, n that household. Aordng to ths alulaton, the Hared populaton reahed about 550,000 by end of As shown n table 2 the new estmate exeeds the ommonly aepted estmate of empral eonomsts by 62 perent. We fnd the populaton estmate of the Soal Survey to be onsstent wth the alulatons of GC, who alulated the sze of the Hared populaton based on revealed (party-) preferene from the 2003 eleton results The BIT Estmate The varable refletng true group membershp and hosen as a benhmark for the ompetng estmates s the sampled person's own delaraton of Hared afflaton (group membershp). In the present ase study ths seems to be the most natural approah sne relgous afflaton s frst of all a subjetve ognton. In other examples of group membershp one mght be lookng for a varable refletng an objetve reognton of group membershp (e.g. a vald passport for tzenshp, a unversty degree for beng an aadem et.) as the benhmark that mght be preferable. Step 1: Searh for an Effent Logst Regresson n the Soure Survey Based on pror knowledge of the dstntly hgh Hared fertlty and fundamental hanges n fertlty over the last generatons we deded to splt the data nto 3 subgroups by the mother's age n order to mprove the overall empral results. In order to dentfy Hared famles we 17 Queston 26: "Do you onsder yourself (1) Hared, (2) relgous, (3) tradtonal-relgous, (4) tradtonal and "not so" relgous, (5) non-relgous or athest. In order to estmate the populaton sze, the detaled data set s needed, nludng nformaton on the other household members, ther age and the weghts attahed to the ntervewed person. These and more data were kndly provded by the CBS. 18 We thank Tsah Makovk from the ICBS for provdng the formula. 19 The total populaton should add up to 6.59 mllon people, but yelds only 6.19 mllon. The dsrepany may be due to the de-fato exluson of the Eastern-Jerusalem Arab populaton, Bedouns n non-reognzed settlements and people stayng n non-sampled nsttutons. Furthermore the weght adjustment reflets only an approxmaton of the true weght. 9

10 analyzed dfferenes n fertlty patterns and demograph haratersts (the mother's age hld rato, ountry of orgn of the head of household), eduatonal haratersts (Yeshva as the last Shool attended by one of the male household members or No Hgh Shool or Unversty dploma), geograph onentraton (areas wth hgh Hared onentraton), dfferenes n soal behavor (phlanthrop behavor, no use of nternet), lvng ondtons (ar ownershp, number of hldren per room). The regresson results for the famles, grouped by the mother's age and the varable defntons are gven n Appendx table 1. The oeffent vetors % ˆ β ( = 1,2,3) are needed for the BIT proess n order to alulate the estmated probabltes by use of the logt funtons g ( x) wth the relevant oeffents for eah group respetvely as mentoned n equatons (1) and (2). The logst regresson model (Appendx table 1) s statstally sgnfant as an be seen from the log lkelhood statst and the Wald-test n table 1. Table 1: Model Sgnfane of the Logst Regresson of Hared Group Membershp (Regresson results from Appendx table 1) LR test Log lkelhood Probablty(LR stat) Wald test Value Probablty Step 2: A foreast for group membershp, ondtonal upon a 'ontnuous' utpont ( ˆ π S ) In the next step we alulate a bnary varable of group membershp from the estmated model based probablty, ondtonal upon the utoff value ˆ π S. Any probabltes exeedng the utoff value reeve a value of 1. All others reeve a value of 0. Ths step s repeated for a near-ontnuous number of utoff values n small steps (say 0.01). The results an then be ategorzed n a lassfaton table suh as table 2 for any spef utoff value. Table 2: Classfaton Table of the Logst Regresson Model at Cutoff Value ˆ π S = 0.5. Classfed Observed (True value) Total 0 1 Total 0 1 Total Total

11 Suh tables measure foreast qualty, as refleted n the alulaton of senstvty (the share of orretly foreasted non-members out of all true non-members), and spefty, (the orretly foreasted members as a share of all true members at any gven utoff value). They are ondtonal upon spef utoff values. 11

12 Table 3: Senstvty and Spefty for the mother's age group at utoff values from 0 1 by nrements of Cutpont Senstvty Spefty 1-Spefty % 0.00% % % 83.58% 16.42% % 90.24% 9.76% % 93.02% 6.98% % 94.36% 5.64% % 94.86% 5.14% % 97.15% 2.85% % 97.43% 2.57% % 97.71% 2.29% % 97.96% 2.04% % 98.06% 1.94% % 98.73% 1.27% % 98.80% 1.20% % 99.08% 0.92% % 99.26% 0.74% % 99.37% 0.63% % 99.47% 0.53% % 99.68% 0.32% % 99.72% 0.28% % % 0.00% Step 3: Assessment of the Foreast Qualty ("goodness-of-ft") n the Soure Survey A well known ndator for the assessment of bnary model foreasts s the ROC (Reever Operatng Charaterst) urve whh juxtaposes senstvty (truly dentfed postve response) wth the perent of outomes wrong postve responses (truly negatve). Fgure 1: The ROC Curve 12

13 Fgure 1 ndates that the youngest age group's regresson performane s best among the 3 groups when evaluated by the ntegral below the urve. Estmates for all 3 groups are better than the 45 o lne of random assgnment. Fgure 1 emphaszes the mportane of splttng up the model estmaton aordng to the mother's age-group, thereby allowng for age-dependent parameter oeffents n the regresson. Step 4: Searhng for the Optmal Probablty Cutoff Value ( π ) After the ROC urves have been alulated, the optmal probablty utoff value needs to be loated among all the possble utoff values. The queston how to translate logst probabltes bak nto a bnary varable s not onlusvely dealt wth n the lterature. In medal researh the queston arses frequently and s sometmes related to the mprovement or the damage n health aused by a spef treatment under revew. For example, f the effet of a ertan vane an only be known ex post and t s found to ause an mportant health mprovement for some, whle for others there s a neglgble negatve effet a general vanaton poly mght be a reasonable ourse of aton. If, lke n the present ase, there s no a pror ase for a partularly large net ost of ether error (FP, FN n seton 3, step 2) we opt for a utoff value that mnmzes the total sum of squared errors FP and FN. Hosmer and Lemeshow (2000) suggest that the optmal utoff value les at the nterseton of senstvty and spefty. Ther hoe seems to hnge on the argument that we attah equal mportane to eah group n a relatve sense. In fgure 2 ths utoff value s at ˆ S,* π = Ths probablty level s surprsngly low, allowng for a great number of mstaken bnary foreasts. The utoff value aordng to the Mnmum-squared-error-rule (MSE) s at ˆ S,* π =0.35, yeldng more relable foreasts of Hared group membershp. The optmalty rule for hosng the utoff value rually affets the number of foreastng errors. In table 4 we ompare the ndene of errors. Whle the MSE-rule redues the number of FP ases sgnfantly, the opposte ours n the FN ases. Ths pattern repeats tself n all three age-groups. However, we also observe that the deteroraton n FN s more than offset by the mprovement n FP. Ths s agan the ase n all three age-groups, suh that we an onlude that the MSE approah meanngfully mproves the foreast, redung the sum of errors to half ompared to the HL approah. ˆ S,* 20 See also Hosmer and Lemeshow, 2000, p

14 Fgure 2: The Optmal Cutoff Value 100% 90% 80% n % of N, Senstvty, Spefty 70% 60% 50% 40% 30% 20% Hosmer-Lemeshow Cutpont BIT Cutpont H=1, HF=0 n % H=0, HF=1 n % Total error n % Senstvty 10% Spefty 0% 0 0,04 0,08 0,12 0,16 0,2 0,24 0,28 0,32 0,36 0,4 0,44 0,48 0,52 0,56 Cutpont Probablty 0,6 0,64 0,68 0,72 0,76 0,8 0,84 0,88 0,92 0,96 1 Table 4: Foreastng Errors n perent of the True Hared n Eah Age Group Foreast True Postve (senstvty) False Postve (FP) False negatve (FN) Sum of Errors (FN+FP) Probablty Cutpont HF % 85.8% 100.0% 193% 60% 100% 170% 46% 100% 186% 63% 100% H1 HF1 H0 HF1 H1 HF0 Age of Female Partner, % 65.6% 13.0% 67.3% 18.5% 33.7% 100.0% - - Age of Female Partner, % 110% 17% 51% 9% 49% 100% - - Age of Female Partner, % 109% 39% 37% 9% 63% 100% - - Total 76% 93% 24% 51% 13% 49% 100% % 52.2% 78.1% 36.3% 167.5% 82.5% 324.3% 170.9% - Hosmer Lemeshow Model Mnmum Squared Error Model Atual Hosmer Lemeshow Model Mnmum Squared Error Model Atual Hosmer Lemeshow Model Mnmum Squared Error Model Atual Hosmer Lemeshow Model Mnmum Squared Error Model Atual 14

15 Step 5: The BIT - Calulaton of the Foreast Hˆ T n the Target-Survey In the fnal step we estmate the Hared populaton by use of the regressons n appendx table 1. Table 5 llustrates the mportane of the qualty of the model to be used for foreastng group membershp. In eah of the observed years the downward bas of the Hared populaton s muh smaller n the reommended approah ompared to the tradtonal approah as used n Berman and Klnov, 1997, or n Dahan, 1998, and elsewhere. Hˆ T Table 5: Alternatve Estmates of Hared Populaton Sze (thousands, perent*, based on data from ) Soure Survey (So.S) Based on the respondents' delaraton , , ,669 0% 0% 0% Optmal BIT , , ,990-14% -33% -33% Eduaton based model 171, , ,966-63% -68% -67% Target Survey (HES) Optmal BIT 361, , , , , , , ,566-16% -21% -38% Eduaton based model 331, , , , , , , ,585-30% -30% -45% Eleton based model (E) 525, ,000 20% *Perentages ndate devatons from the populaton sze based on the respondents' delaratons Soure: Soal Survey, Household Expendture Survey, Eleton results, Central Bureau of Statsts. 5. Conludng Remarks Ths methodology an be usefully appled to many felds, sne t allows us to optmally enhane a gven data base by addng a bnary varable that does not exst n the relevant data base. The present study shows how our methodology an be used for poverty estmatons of a populaton subgroup that s not sampled n the major surveys used for poverty alulatons the nome survey or the expendture survey. The man results are reported n table 6. Poverty ndene n the Hared populaton s nearly three tmes hgher than n the general populaton. It s among the poorest populaton groups n Israel, makng t obvously hghly neessary to montor efforts of poverty reduton. Inequalty among the poor, though t s lower than n the general populaton, does not ompensate for the hgher poverty ndene and nome gap. Poverty ntensty, as measured by the Sen-ndex s almost double ts level for the total populaton. A smlar onluson an be drawn onernng hld poverty. 15

16 Whle untl reently Hared poverty has been approxmated only roughly, the present methodology mproves the auray of poverty measurement for the targeted group, a desrable feature, the more expensve and the longer the tme lag of poly mplementaton. 21 Table 6: Relatve Poverty n Israel among the Hared and the Total Populaton 22 1/2 Medan Equvalzed Poverty Hared Populaton Total Populaton Headount 60% 23% Gn Index of the Poor Inome Gap 31.6% 33.3% Sen Poverty Measure Chld Poverty Headount 63% 33% Hared Populaton Sze 409,566 6,274,115 Soure: Expendture Survey, 2004, C.B.S The proposed method may also be usefully appled n the ontext of poverty mappng by Small Area Estmaton (see for example Hentshel et al., 2000). In suh an exerse we mght be nterested n attahng a bnary foreast of poverty ndene to a household n a small area, not overed by the household surveys typally used for offal poverty alulatons (the Soure Survey, S). Data on households n the small area, olleted n an extensve but superfal large sale survey suh as a survey aompanyng a populaton ensus ould be used as the vetor x' n a Target Survey, T, as outlned n setons 2 and 3. Use of the ondental vetor x' n S and T and the hoe of an optmal utoff value would allow for the produton of an estmate of poverty ndene n T. Referenes Berman El and Ruth Klnov, 1997, "Human Captal Investment and Nonpartpaton: Evdene from a Sample wth Infnte Horzons (Or: Mr. Jewsh Father Stops Gong to Work)," Jerusalem, The Maure Falk Insttute for Eonom Researh n Israel, Dsusson Paper No , p Berman El, 2000, "Set, Subsdy, and Sarfe: An Eonomst's Vew of Ultra-Orthodox Jews", The Quarterly Journal of Eonoms, August, p Bgman Davd and P.V. Srnvasan, 2002, "Geographal Targetng of Poverty Allevaton Programs: Methodology and Applatons n Rural Inda", Journal of Poly Modelng, 24, Dahan Mom, 1998, "The Ultra-Orthodox Jews and Munpal Authorty, Part 1 Inome Dstrbuton n Jerusalem", n Hebrew, The Jerusalem Insttute for Israel Studes, Researh Seres No. 79, Jerusalem, p See Glewwe and Van der Gaag, Poverty n the Hared populaton, based on an earler verson of the BIT-methodology s analyzed n detal n Gottleb,

17 Degan Av and Rna Degan, 2000, "The Demand for Housng n the Hared Setor", Insttute for Spatal Analyss Ltd., September, p Flug Karnt and Ntsa (Kalner) Kasr, 2003, "Poverty and Employment and the Gulf between them", Israel Eonom Revew, Vol. 1, p Fredman Menahem, 1991, "The Hared (Ultra-Orthodox) Soety Soures, Trends and Proesses", n Hebrew, Summary n Englsh, The Jerusalem Insttute for Israel Studes, Jerusalem. Glewwe Paul and Jaques Van der Gaag, 1990, Idenfyng the Poor n Developng Countres: Do Dfferent Defntons Matter? World Development, 18 (6), Gottleb Danel and Ntsa Kasr, 2004, "Poverty n Israel and a Strategy for ts Reduton", n Hebrew, The Bank of Israel, July, Gottleb Danel and Roy Manor, 2005, "On the Choe of a Poverty Measure: The Case of Israel, 1997 to 2002", n Hebrew, Abstrat n Englsh, forthomng, The Bank of Israel, Marh, Gottleb Danel, 2006, "Poor and Trapped n Ideology: A Case-Study of Poverty n the Jewsh Ultra-Orthodox Soety n Israel", (n Hebrew) forthomng, Van Leer Insttute, Jerusalem, Gurovh Norma and Elat Cohen-Kastro, 2004, "Ultra-Orthodox Jews Geograph Dstrbuton and Demograph, Soal and Eonom Charatersts, ", n Hebrew, Summary n Englsh, Workng Paper Seres, No. 5, July, Central Bureau of Statsts Demography Setor. Hadjostas Petros and George C. Hadjnola, 2001, "The Asymptot Dstrbuton of the Proporton of Corret Classfatons for a Holdout Sample n Logst Regresson", Journal of Statstal Plannng and Inferene, Vol. 92 (1-2), January, Hentshel, J., J.O. Lanjouw, P. Lanjouw and J. Pogg, 2000, "Combnng Survey Data to Trae the Spatal Dmensons of Poverty: A Case Study of Euador", World Bank Eonom Revew, 14, (1), Hosmer Davd, W. and Stanley Lemeshow, 2000, Appled Logst Regresson, 2nd Edton, John Wley & Sons In., New York, USA. Mayshar Yoram, 2004, "Potental Inome as a Measure of Poverty n Israel", forthomng n the Israel Eonom Revew, (n Hebrew), p Shares G., et al., 2003, "Regonal Dstrbuton of Bovne Neospora Cannum Infeton n the German State of Rhneland-Palatnate Modeled by Logst Regresson", Internatonal Journal of Parastology, Vol 33 (14), Deember, Shutter E.M.J. et al., 1998, Estmaton of Probablty of Malgnany Usng a Logst Model Combnng Physal Examnaton, Ultrasound, Serum CA 125, and Serum CA 72-4 n Postmenopausal Women wth a Pelv Mass: An Internatonal Multenter Study", Gyneolog Onology, Vol. 69, (1), Aprl, Stegeman, J.A. et al., 2006, "Establshng the Change n Antbot Resstane of Enteroous Faeum Strans Isolated from Duh Brolers by Logst Regresson and Survval Analyss", Preventve Veternary Medne, Vol 74, (1), Aprl,

18 Appendx Table 1: Logst Regresson for Hared Afflaton Prob. Coeffent Prob. Coeffent Prob. Coeffent Varable C AC15_ LSY DIST_ DIST_ PHILANT IL_HH_m CH_ROOM CAR NO_DIPL INTERNET The varable lst: AC15_2 Bnary varable ndatng rato between number of hldren n household and the age of the mother at values LSY Bnary varable ndatng, that the last shool attended by any of the male members of the household was a relgous semnar (Yeshva). DIST_11 Bnary varable ndatng that the household was sampled from the Jerusalem dstrt (1,0). DIST_51 Bnary varable ndatng that the household was sampled from the Tel-Avv dstrt (1,0). PHILANT Bnary varable ndatng phlanthrop atvty by head of household (1,0). IL_HH_m Bnary varable ndatng ountry of brth of household head as Israel (1,0). CH_ROOM The number of hldren dvded by the number of rooms, n the household. CAR Bnary varable ndatng ar ownershp (1,0). NO_DIPL Bnary varable ndatng that head of household never got any shool/unversty dploma (1,0). INTERNET Bnary varable ndatng household's use of nternet (1,0). 18