CREDIT SCORING APPROACHES GUIDELINES

GLOSSARYAlternative dataOverfittingInformation gathered from nontraditional datasources. Examples may include geolocation data,point-of-sale transactions, device data, and socialmedia posts.Scenario where the analysis corresponds too closelyto a particular set of training data, resulting in afailure to predict future observations accurately.Cascading risksIllustration of a summary of the trade-off betweenprecision (y-axis) and recall (x-axis) at differentprobability thresholds.Scenario where a single error is amplified or leadsto a chain reaction in downstream processes owingto the interconnectedness of systems.Credit ratingPrecision-recall curvesPrecisionNumerical expression representing the creditworthiness of an entity.Measure of the relevancy of a result. It is calculatedas the number of true positives divided by the sumof the number of true positives and false positives.Credit scoringRecallForm of statistical analysis that provides an estimateof the probability that a loan applicant, existingborrower, or counterparty will default or becomedelinquent.Measure of the relevancy of how many trulyrelevant results are returned. It is calculated as thenumber of true positives divided by the sum of thenumber of true positives and false negatives.Credit reporting service providerReceiver operating curveEntity that administers a mechanism enabling creditinformation collection, processing, and furtherdisclosure to users of data as well as value addedservices based on such data.Graphical plot that represents the diagnostic abilityof a binary classifier as its discrimination thresholdvaries. It is created by plotting the true positive rateagainst the false positive rate at different settings ofthreshold values.Credit services providerEntity that provide loans and other forms of creditto consumers and businesses. It includes financialinstitutions, banks, financial technology providers,and alternative lenders.Semistructured dataForm of structured data that does not conformwith the structure of data models associated withrelational databases. It contains tags or othermarkers to enforce hierarchies of records and fieldswithin the data.CREDIT SCORING APPROACHES GUIDELINESIX

XStructured dataUnstructured dataAny data that reside in a fixed field within a recordor file. Typically, the data reside in the form ofrelational databases and spreadsheets. The formalstructure allows one to easily enter, store, query,and analyze the data.Data that do not have a predefined data model or arenot organized in a predefined manner. They existtypically in the form of text files, images, socialmedia data, and sensor data.GLOSSARY

POLICYRECOMMENDATIONSPolicy Recommendation 1: A legaland ethical framework is required togovern and provide specific guidanceto credit services providers (CSPs).Where not done yet, regulatory bodies should putin place a legal and ethical framework that providesfor appropriate oversight and responsible use ofcredit scoring. An ethical framework upholdsfundamental human rights and ethical principles andincorporates a CSP’s values and codes of conduct,while also ensuring that the legitimate interests ofthe CSPs can be met. The framework should alsoconsider the protection of consumer rights and dataprivacy aspects.Policy Recommendation 2: Thedecisions made on the basis ofcredit scoring should be explainable,transparent, and fair.CSPs should understand and be able to explain toconsumers the lending decisions made on the basisof credit scoring. CSPs should be able to understandand explain to regulatory bodies the way creditscoring is incorporated into their processes andthe logic involved in its functioning. The dataused, and the decisions made on the basis of creditscoring, should operate within equal opportunity oranti-discrimination laws (for example, to not usecharacteristics considered protected such as raceand religion).Explainable and Transparent CreditScoring Decisions for ConsumersConsumers should receive enough information onthe data used and the decisions made on the basis ofcredit scoring methods. The focus should, however,not be on the direct or indirect disclosure of thealgorithm, but rather on the rationale behind thecredit risk decision. Disclosure of the algorithmmay infringe on proprietary rights, could leadto compromising its accuracy, and also may notbe meaningful to the consumer. Organizationsshould consider providing the data subjects with anavenue to request a review of decisions that werefully automated and a correction of underlyinginaccurate data (if this resulted in their credit scorebeing impacted).There is also a need to be transparent to consumersabout the data collection process. The mechanismsshould provide consumers with the key facts aboutdata origin, the potential users and uses, any disputeresolution mechanisms, and the lawful use ofpersonal data. If the data are used for a purpose otherthan that specified during data collection, withinthe boundaries of country-specific legislation, thelawful collection of data is required. If traditional oralternative data are used, consumers and regulatorsshould have the right to know the source fromwhere the data were extracted. The guidance alsoapplies to cross-border data flows.CREDIT SCORING APPROACHES GUIDELINESXI

Explainable and Transparent Use ofCredit ScoringCSPs should be able to explain to regulators theway the credit scoring is incorporated into theirprocesses and the logic involved in its functioning.CSPs should be able to quantify, explain, and adoptadequate measures to mitigate unintentional andpotentially amplified risks associated with the useof credit scoring methods when appropriate and doso proportionately to the magnitude of the risks.In cases where algorithms are not easily explainable(yet are parsimonious and justifiable), additionalsteps should be taken to verify that the input data,algorithms, and outputs are performing withinexpectations.Policy Recommendation 3: Dataaccountability practices should bestrengthened.Recognizing the increasing risks of cyber security,data privacy, and consumer protection violations,owing to digitalization, industry participantsshould put in place sufficient controls to ensuredata integrity and privacy. CSPs should understandthe source of data and the way data are collected,updated, and improved over time.Security of DataCSPs should put in place sufficient controlsand preventive measures against cyber attacks.Regulatory authorities should consider puttingcyber laws in place that include punitive measuresin cases of data breaches. There is a need for regularrisk assessments, swift response, and reportingof incidents occurring in-house or at outsourcedservices providers.Data Privacy and Consumer ProtectionCSPs should integrate data privacy into the designprocess when building credit scoring methods.Privacy impact assessments will ensure that personaldata are not used unlawfully without the permissionXIIPOLICY RECOMMENDATIONSof the consumer. In addition, the consumer shouldbe accorded the rights to correct, to object to the useof their personal data for specific purposes, and totransfer or request deletion of information, subjectto country-specific legislation.Accountability in Data UsageCSPs should specify the sources of data usedfor credit scoring in accordance with the rightsof consumers and businesses. This practice isespecially important in the cases of third-party datasources and alternative data. There should be clearaudit trails on the use of data for credit scoring; thedata flow from the original source needs to be clearlyidentified. This approach may include checks fortypes of data (proprietary or nonproprietary) andthe accuracy of the data used.Policy Recommendation 4: Creditscoring models should be subject to amodel governance framework.Credit scoring models, developed using bothtraditional and innovative techniques, should besubject to an effective model governance frameworkthat considers, but is not limited to, the managementof model risk, including the conceptual soundness ofthe model; assessment of unintended consequencessuch as cascading risks and the disregard ofprotected characteristics (for example, race, genderand religion); model ownership within a businesscontext; and regular reviews and back-testing ofmodels, including validation of model performancesuch as receiver operator characteristic (ROC)curves and/or precision-recall (PR) curves.Policy Recommendation 5:Collaboration and knowledge sharingshould be encouraged.Sharing of DataTo the extent possible, collaboration betweenindustry, governments, and regulatory bodies shouldbe encouraged. Access to data plays a vital role in

innovation and should be encouraged by regulatorybodies. To this end, regulatory authorities shouldconsider implementing collaborative models suchas open banking as a way of fostering innovation.Public policy should broadly encourage wider andtimely availability of private and publicly held data,while ensuring full protection of personal data.Sharing of Knowledge amongStakeholdersIn addition to increased investments in researchand the study of significant trends, regulatorybodies should publish their research and encourageindependent study by industry and academicbodies to increase knowledge and understandingand deepen research on ethical aspects andtransparency of innovative credit scoring methods.The collaboration and sharing of best practicesamong industry peers may help create and fosterindustry best standards.Financial literacyThere is a need to ensure that consumers areeducated and made aware about the uses of creditscores. Financial literacy, numeracy, and capacitygo hand in hand with financial inclusion.Policy Recommendation 6: Theregulatory approach should strike abalance between innovation and risk.Regulators need to strike a balance betweenharnessing the opportunities presented by creditscoring and mitigating risks. Financial integrityand data privacy should be protected. For example,consumers need to be protected from discriminatorydecisions, while innovation should be encouraged.Improved practices in risk assessments mayimprove the accuracy of the risk weighting ofassets and aid supervision. In addition, the impactof regulations on the development and adoption ofinnovative credit scoring and financial inclusionshould be carefully considered.Policy Recommendation 7: Capacitybuilding of regulatory bodies andwithin CSPs is essential.Regulatory bodies should embrace an openness tochange and an awareness that attitudes and practicesare still evolving. Concretely, they should establishframeworks for innovation, such as regulatorysandboxes to support organizations that aredeveloping products and services that use personaldata in innovative ways, while also ensuring a safeand protected testing environment.There should also be measures for capacity buildingand skills development within Regulatory bodies,in order to understand and supervise models andnew innovations.There is a need for heightened open and transparentcommunication that will help develop trust andconfidence in the adoption of new technologiesMeanwhile, rules need to be in place to ensuretransparency and consistency.Furthermore, the organizational setup, resources,and infrastructure capacity should be facilitatedwithin CSPs. Dedicated focus on credit scoringshould be supported with sufficient experts andtechnology. Coherent and well-connected workinggroups should be part of the development andassessment process of credit scoring methods.CREDIT SCORING APPROACHES GUIDELINESXIII

1.BACKGROUNDIntroductionThe uses of credit scoring methods have increasedsignificantly in recent years, owing to access to data,rise of computational power, regulatory requirements,and demand for efficiency and economic growth(Demirguc-Kunt, Klapper, and Singer 2017).Furthermore, the application of credit scoring hasevolved from the traditional decision making ofaccepting or rejecting an application for credit tothe inclusion of other facets of the credit processsuch as the pricing of financial services to reflectthe risk profile of the consumer, setting of creditlimits and regulatory capital, customer relationshipmanagement, and, in certain countries, targeting ofprospective customers with offers.In some cases, the use of sophisticated credit scoringmethods has increased from traditional statisticaltechniques such as linear discriminant analysis andlogistic regression to innovative methods such asartificial intelligence, including machine learningsuch as random forests, gradient boosting, and deepneural networks.The adoption of innovative methods has increasedin many cases not only the sophistication of thecredit scoring methods but also their opaqueness.Unlike traditional credit scoring models, innovativemethods are often viewed as challenging to interpretand explain (FSB 2017). In addition, innovativemethods are prone to overfitting (that occurs whenthe analysis corresponds too closely to a particularset of training data, resulting in a failure to predictfuture observations accurately) and raise concernsabout fairness and discrimination against minorities(European Commission 2018b).The adoption of alternative modelling techniqueshas also broadened the range of data that could beconsidered relevant for credit scoring models anddecisions. For example, credit services providers(CSPs) are now leveraging nontraditional datasources to score consumers and businesses withlimited credit bureau information. The use ofalternative data, such as granular transactionaldata in decision processes, however, has arousedincreased interest from data privacy advocates.Likewise, regulators have taken a keen interest in theapplication of credit scoring, because of its potentialimplications for national financial systems and thebroader goal of financial inclusion.There are concerns about the effectiveness ofcredit scoring and technologies. This is especiallytrue in markets where there is little or no adequateregulatory oversight or industry codes to regulatethe conduct of CSPs.Evolution of Credit ScoringIn the early days of credit reporting, CSPs used creditreports to offer financial services to consumers,businesses, and large corporations. Credit reportsprovided information about the consumer orbusiness’s demographics, insurance, and otherutilities (Aire 2017).The scientific background to modern credit scoringwas pioneered by the statistical technique ofdiscriminant analysis, devised by Ronald. A. Fisher(Fisher 1936). Discriminant analysis is a statisticaltechnique used to differentiate between groups ina population through measurable attributes whenthe common characteristics of the members of theCREDIT SCORING APPROACHES GUIDELINES1

group are unobservable. In 1941, Durand recognizedthat the same approach could be used to distinguishbetween good and bad loans.One of the first credit scoring algorithms wasdeveloped using linear programming (myFICO2018). Initially, both the variables selected and thescores assigned were mainly judgmental. However,the systematic application of the scoring methodcontributed to consistency in the credit applicationsprocess. This approach became the start of usingstatistical methods to determine creditworthiness inan organized and transparent manner.Small credit reporting companies, referred to ascredit reporting service providers (CRSPs) in thisguideline, developed into larger organizations thatkept more accurate information. In 1970 in theUnited States, the Fair Credit Reporting Act (FCRA)was passed requiring CRSPs to open their files tothe public; ensure discriminatory data includingrace, gender, and disability are not used for creditdecisions; and delete negative information after aspecified period (Federal Register 2011). Overall,computer technology, market forces, and the FCRAprovided CRSPs with the impetus to transformthemselves from small cooperatives to large scaleCRSPs (Furletti 2002).CRSPs can exist as either credit bureaus or creditregisters. Credit bureaus are typically privately ownedcompanies that collect information from financialand nonfinancial entities, including microfinanceinstitutions, and provide credit information to CSPs.Credit registries tend to be public entities managedby supervisors or central banks (World Bank 2016a).Ultimately, the economics of processing a highvolume of loan applications, along with theimprovement of the predictive power of the modelsand the constant advances in available computingpower, lead to the acceptance of statistically based,automated scoring systems worldwide.Credit scoring methods that use innovative algorithmsare designed to speed up lending decisions, whileassessing risk more accuratel—CSPs have long21. BACKGROUNDrelied on credit scores to assess risk when makinglending decisions for consumers and businesses.Historically, to capture the willingness and ability ofthe borrower to repay, data on past payment historyserved as the foundation of most credit scoringmodels (Federal Register 2011). These models havetraditionally been developed using methods suchas regression, decision trees, and other statisticalanalyses to generate a credit score using limitedamounts of structured data. However, in somecases, CSPs and CRSPs are increasingly turning toadditional, unstructured, and semistructured datasources, including data sources such as open bankingtransactions (see, for example, PSD 2, the revisedPayment Services Directive [European Commission2015]) and data obtained from mobile phone useand other digital sources, in an effort to capture aricher and more granular view of an applicant’screditworthiness and improve the accuracy of models(Sidiqqi 2005). In markets that use credit scoringmodels based on traditional data sets, a potentialborrower is required to have enough historical creditinformation available to be considered scorable.In the absence of this information, a credit scorecannot be generated, and a potentially creditworthyborrower is often unable to gain access to lending inacceptable conditions.By combining innovative algorithms and newdata, a much more detailed assessment of thecreditworthiness of consumers and businesses ispossible. In essence, the world is changing and newdata sources are being created. However, challengesremain as the models and important variablesdeveloped with techniques such as machine learningon new data sources may be difficult to interpret andmay also require additional testing.In addition to facilitating a potentially more precise,segmented assessment of creditworthiness, the useof innovative algorithms in credit scoring may helpenable greater access to credit. In summary, usingalternative data sources and applying innovativealgorithms to assess creditworthiness, CSPs maybe able to arrive at credit decisions that previouslywould not have been possible.

Credit Scoring DefinitionsCredit ScoringCredit scoring is a statistical method used to predictthe probability that a loan applicant, existingborrower, or counterparty will default or becomedelinquent. It provides an es

credit scoring include greater financial inclusion and access to credit, improvement in the accuracy of the underlying models, efficiency gains from the automation of processes, and potentially an improved customer experience. The use of innovative methods for credit s