1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other .
1.LOK(F/H)Q2.LOD(1-5)3. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnly6. Source(B/ M / N)7. Status(U /E /S)8. Explanation1F2NSD, 41.7, CR2H3NSA, 41.7, CR3F3MSD, 41.10, CR, 1998 NRC ExamC, 41.10, CRNeed to add OP-2203.002 to the stem and makepart 2 of the answers consistent (i.e. based on )Will make the change.The question was edited as recommended.Question SatB, 41.10, CRThe reference (Step 6) used in the answerjustification for the 10-90% level is a contingencyaction for verifying natural circulation. Step 8discussed the reasons for 10% and 90%.Discussion on Answer C is confusing due to thelimit of 90% is for water carryover in bothshutdown and power operations. Recommendrewording the justification.Will make the change.The justification was revised.Question SatNESNES3BSC, 41.5, CR, 2005 and 2006 NRC ExamH4NSA, 41.7, CRF2NSDC, 41.7, CRChanged order of the answers.Question Sat4H35F36H78EEInstructions(Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:1.2.3.4.5.6.Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)Check the appropriate box if a psychometric flaw is identified: “Stem Focus”: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). “Cues”: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.). “T/F”: The answer choices are a collection of unrelated true/false statements. “Cred. Dist”: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. “Partial”: One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).Check the appropriate box if a job content error is identified: “Job Link”: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). “Minutia”: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). “# / Units”: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). “Backward”: The question requires reverse logic or application compared to the job requirements.Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable)Enter question’s source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f.
7.8.Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).1.LOK(F/H)2.LOD(1-5)9H310F311H3Q3. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/AESROOnly6. Source(B/ M / N)7. Status(U /E /S)NSNESBS8. ExplanationC, 41.2, CRModified stem following validationQuestion SatA, 41.7, CRCueing in Answer A. The stem says you have atube rupture in 2 of the three given conditions andthe correct answer actually says ruptured SG.Maybe use minimize RCS break flow Will make the change.The question was revised.Question SatD, 41.7, CRB, 41.4, CRReferences discuss Main Feed Reg Valve bypassopening instead of MFWP Recirc Valve closing.Confusing did you want to discuss the operationof the MFWP Recirc valve or the MFRV Bypassvalve. Think the KA follows more with the MFRVBypass valve operations.Will up-date the references.The references were revised.Question SatAC, 41.10, CRModified Question due to change in EOPidentified during validationQuestion SatA, 41.7,Answer B contradicts itself. The first part of theanswer aligns SW to the suction of EFW and thesecond part states that it minimizes the possibilityfeeding the SGs with SW.Will replace half of the two by two.This question was revised.Question SatNESNSNUS3BSD, 41.8, CR, 2009 NRC ExamF2NSA, 41.7, CR17H3MSB, 41.4, CR, 1998 NRC Exam18H3MSB, 41.4, CR, 2012 NRC and 2013 Audit Exams19H3BUSCA, 41.7KA mismatch – does not test the applicant’sknowledge of operating/monitoring RPI following12F213H314F215H16EEUU
1.LOK(F/H)Q2.LOD(1-5)3. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnly6. Source(B/ M / N)7. Status(U /E /S)8. ExplanationCRW.This question is not a KA mismatch as written –CEAC is used as part of the RPI.Modified stem and answers to address commentsduring validation. Removed extra informationfrom answers that was not required.Correct answer now AQuestion is SatB, 41.5,KA mismatch – does not test the reason for theTS limits for response to a dropped rod.Will reword part of the stem to correct.This question was revised.Question SatA, 41.5KA mismatch – question does not test knowledgeof the comparison of makeup and letdown flow.This question is Sat as written.20H2UNUS21H3UNUS22H3NSD, 41.7, CR23F3BSA, 41.7, CR, 2000 and 2005 NRC Exam24H2NSC, 41.10, CR25H3NSC, 41.7, CR26H3NSD, 41.5, CREditorial corrections in response to commentsmade during validation.Question Sat27F3NSB, 41.7, CR28F3BSB, 41.10, CR, 2000 and 2011 NRC ExamEditorial corrections in response to commentsmade during validation.Question Sat29H3NSB, 41.3, CR30F3NSC, 41.3, CR31F3MSD, 41.7, CR, 2012 NRC Exam32H3NSA, 41.5, CRBESD, 41.3, CR, 2012 NRC and 2013 Audit ExamsAdd the procedure reference (2103.002) in thestemWill add the procedure reference to the stem.The procedure reference was added to the stem.33F3E
1.LOK(F/H)Q2.LOD(1-5)3. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnly6. Source(B/ M / N)7. Status(U /E /S)8. ExplanationQuestion SatD, 41.11, CRDelete “may be” from the last line of the stem.Will change.This question was edited.Question SatA, 41.4, Reference discussed in support of thecorrect answer is not contained in the documents.OP-2104.028 Section 20.0Will provide the reference.Reference providedQuestion SatMES3NSF3NSB, 41.7, CRF2NSB, 41.6, CR34H335F363738H339H340F241H342F43ED, 41.7, CRAnswer justification needs to be corrected. List Das correct and incorrect.Justification correctedQuestion SatA, 41.7, CRIt could be argued distractors ‘A’ and ‘B’ are bothcorrect (i.e. the overloads may be reset after thecause is determined. Recommend adding“ONLY” to distractors ‘B’ and ‘D’.Will make the recommended changes.This question was revised as recommended.Question SatMSNESNSD, 41.4, CRBSC, 41.10, CR, 2012 NRC ExamBank question since it is not significantly changedfrom original question.Question Sat3NSA, 41.7, CRH3NSA, 41.4, CR44H3NS45F3NS46F3NSEC, 41.5, CRModified question in response to commentsduring validation.Question SatA, 41.7, CRModified question in response to commentsduring validation.Question SatB, 41.4, CR
1.LOK(F/H)Q2.LOD(1-5)3. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnly6. Source(B/ M / N)7. Status(U /E /S)8. ExplanationB, 41.4, CR, NRC 2011KA mismatch because the question asks theeffect of the DC system on the AC system ratherthan the effect of the AC system on the DCsystem.This question is Sat as written.C, 41.7, CR, 2006 NRC ExamBoth C and D are correct answers sinceobserving the voltage or maintenance testingcould determine the severity of the ground.The answer justifications are incorrect for theanswers given. Need to provide correctjustifications.Will update justifications. Will revised the seconddistractor.This question was revised.Question SatC, 41.8, 1998 NRC ExamWhat is the reference for the loads off of2Y13/24?Load is spelled wrong in the stem.Will correct spelling and add a reference.Still need to change “load” to “loads” or “loading”in stem for it to read right.The question was revised.Question SatBUSBESBES3BSB, 41.8, CR, 2009 and 2014 NRC Exam3MSD, 41.7, CR, 2002 and 2008 NRC , 41.12, CRIt could be argued there is no correct answersince there is more than one required action tocontinue discharging. Recommend changingstem to read something like “Which of thefollowing is a REQUIRED action to continue therelease ”Will revise the question.This question was revised as recommended.Question SatB, 41.3, CRKA mismatch – The question does not addressthe ability to predict or monitor changesassociated with operating the SWS controls.The cooldown rate is controlled through the use ofSWS controls – this question is Sat as written.B, 41.4, CRAnswer A and B would both be correct from thegiven stem. Recommend change stem to readsomething like “remain closed for a minimum ofminutes.”Will revise the question.
1.LOK(F/H)Q2.LOD(1-5)3. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnly6. Source(B/ M / N)7. Status(U /E /S)8. ExplanationThis question was revised as indicated.Question Sat55F3BSB, 41.9, CR, 2008 NRC Exam56H3NSC, 41.6, CRCD, 41.3, CRMissing reference to support the flange levelWill provide the reference.Reference providedRearranged answers and modified correctanswer.Question SatB, 41.6, CRModified answers to better align with AOPQuestion Sat57H3NS58H3NS59F3BSA, 41.7, CR, 2006 NRC Exam60F2BSD, 41.9, CR, 2011 NRC and 2012 Audit ExamBUSNES3NS3BS61F362H363H64HUEEB, 41.13, CR 2005 NRCSRO only question due to requiring knowledge ofthe fuel handling equipment.Answer A could also be considered correct if thehoist cable is considered part of the fuel grapplingassembly.Will replace the KA.The KA and question were replaced.Question SatD, 41.4, CRStem stating that the vacuum is degraded due toair in-leakage cues that answer D is correct.Need to rework the stem. Placing a loweringvacuum in the conditions and asking for whatactions are required to restore condenservacuum. That would have the applicant assessthat the lowering vacuum is due to air in-leakageinstead of being told as part of the stem.Handswitch in the first sentence in the Notessection needs the “n” removed from the end.Will revise question.This question was revised.Question SatC, 41.13, CRSuggest changing answer B to same structuralformat as other three answers.Distractor ‘B’ was revised.Question SatA, 41.11, CR, 2012 NRC Exam
Q651.LOK(F/H)2.LOD(1-5)H33. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnly6. Source(B/ M / N)N7. Status(U /E /S)S8. ExplanationD, 41.4, CRC, 41.10, CR, 2014-2 NRC ExamNeed to correct the stem: and the MAXIMUM . requirement is within a This question was edited as recommended.DC, 41.10, CRModified stem and answers to remove SROaspectsQuestion SatB, 41.10, CRJustification notes need to annotate the correctanswer as Answer B.Corrected notesQuestion Sat66F2MS67H2NS68F2NS69H3BSA, 41.10, CR, 1998 NRC Exam70H3NSD, 41.10, CR71F3BSD, 41.13, CR, 2006 NRC ExamBank question due to the question was notsignificantly modifiedQuestion Sat72H3BSC, 41.13, CR, 2012 NRC Exam73F2BSA, 41.8, CR, 2012 NRC Exam74F275H376(1)H377(2)H3EUBESBSNUSBSC, 41.10, CR, 1998 and 2008 NRC ExamsEdit, make the stem read “ MAXIMUM allowedinterval ”Will revise the question.This question was revised as recommended.Question SatD, 41.7, CR, 2014 NRC ExamBank question due to the question was notsignificantly modifiedQuestion SatBA, 41.5, CRThis is an RO question because it can beanswered using systems knowledge and alarmcard actions. May want to consider selecting anew KAWill revise the question.This question was revised.Question SatB, 43.5, CR, 2012 NRC ExamBank question due to the question was notsignificantly modifiedQuestion Sat
Q1.LOK(F/H)2.LOD(1-5)78(3)H379(4)H380(5)H33. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. H36. Source(B/ M / N)7. Status(U /E NS87(12)H4NS8. ExplanationBD, 43.2, CR, 2012 NRC ExamIf answer B is correct then answer D would becorrect in stating that the RCS is capable ofaccommodating pressure surges during alltransients.Will revise the question.This question was revised.Question SatC, 41.7, CRThis is an RO question since it tests on AOP entryconditions and associated immediate operatoractions. It is a KA mismatch because it does nottest on EOP interlocks, etc, associated with EOPentry conditions.This question is Sat as written.A, 43.2, CRD, 41.10, CRThis is an RO question since it tests on AOP entryconditions and associated immediate operatoractions.This question will be revised.Replaced question with modified bank questionQuestion SatDB, 41.7, CRRo system knowledge only required to answerquestionThis question is ok as written as it is from acontingency procedure.Modified question due to overlap with audit examJPM.Question SatB, 43.7, CR, 2012 NRC ExamB, 41.10, CRThis is an RO question since it tests on AOP entryconditions and associated immediate operatoractions.This question is Sat as written as it is from acontingency procedure.C, 41.10, CRThis is an RO question since it tests on AOP entryconditions and associated immediate operatoractions.This question is Sat as written.C, 41.10, CRModified stem following validation.Question SatB, 43.2, CR
3. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnly6. Source(B/ M / N)7. Status(U /E H4MESE8. ExplanationC, 43.2, CRJustification notes have answer C as correct andincorrect. Need to change the incorrect Cjustification to DJustification correctedQuestion SatA, 43.2, CR, 2014 NRC ExamThis is an RO question since it tests on AOP entryconditions and associated immediate operatoractions.This question is Sat as written.D, 41.10, CRThis is an RO question since it tests on AOP entryconditions and associated immediate operatoractions.This is an SRO question since the transition to theSPTAs is in the AOP making it a subsequentEOP.Question SatD, 41.10, CRThis is an RO question since it tests on AOP entryconditions and associated immediate operatoractions.This question will be revised.This question was revised. Change the cue toread “the CRS would transition to .” Instead of“the CRS would use .”The question was revised.Question SatAB, 43.5, CRBased on the references provided, distractors ‘A’and ‘B’ both appear to be correct depending onwhat procedure (the ACA or the OP) isreferenced.Will revise this question.This question was revised.Question SatC, 43.2, CRAnswer A and B should be changed to “be placedin Level Cal”.We do not expect applicants to have the tablesassociated with TS memorized. Recommendproviding TS 3.3.1 as a reference.Will revise this question.This question was revised.Question SatC, 43.5, CRIn order to correctly answer this question, theapplicant would have to have the procedurememorized, including the tables defining thedifferent fire zones. This is not a realisticexpectation. If the reference is provided, then itbecomes a direct lookup.
)F499(24)F3100(25)F33. Psychometric FlawsStemFocusCuesT/F4. Job Content FlawsCred.DistPartialJobLinkMinutia#/Units5. OtherBackwardQ–K/ASROOnlyUUE6. Source(B/ M / N)7. Status(U /E /S)BMUSNS8. ExplanationSimilar to question 90 for fire actions.Recommend changing the questionWill revise this question.Replaced with modified bank question.Question SatA, 41.7, CR, 2011 NRC ExamThis is an RO question since it is testing systemsknowledge. It is a KA mismatch because this is atier 3 question and it does not have a genericapplication. Bank question was a previously usedRO question.Will change the reference to ts bases.New KA selected. Replaced question withmodified bank question. Change cue to “requireprompt transition into .” from “required promptentry into .”Question SatC, 43.2, CRNUSDA, 43.2. CR, 2009 NRC ExamKA mismatch because this is a tier 3 question andit does not have a generic application. Previoususe was a tier 2 question. In addition, we do notexpect applicants to have 24 hour actionstatement memorized.Will revise the question.Selected new KA and replaced with new questionQuestion SatBSA, 43.7, CRMESD, 43.5, CR, 2009 NRC ExamNeed to change the second bullet in the stem toReactivity Control safety function instead ofReactivity safety function.Will revise the question.This edit was made as recommended.Question SatBSA, 43.5, CR, 2012 NRC Exam
ES-401Written Examination Review WorksheetRO TOTALS:B 25 (33%)F 37 (49%)E 13 (17%)M 8 (11%)H 38 (51%)U 3 (4%)B 5 (20%)F 5 (20%)E 5 (20%)M 6 (25%)H 20 (80%)U 5 (20%)Form ES-401-9Additional Notes:N 42 (56%)SRO TOTALS:Additional Notes:N 14 (56%)GENERAL COMMENTS:1.Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N2.Chief Examiner follow-up comments are indicated in blue.3.Average difficulty is 2.79 on the RO exam anddifficulty.4.The 10CFR55.41/43 distribution is: RO / SRO41.1 043.1 041.2 143.2 741.3 543.3 041.4 1043.4 041.5 643.5 541.6 343.6 041.7 2443.7 241.8 441.9 241.10 13/141.11 241.12 341.13 241.14 05.The answer distribution is:A 19 / 8B 20 / 5C 18 / 7D 18 / 56.There are 2 /3.20on the SRO exam. These values for this reviewer indicate the RO exam is average difficulty and the SRO exam is higher than averagequestions with attachments provided.NUREG 1021, Revision 10
(1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Backw ard Q – K/A SRO Only Question Sat 34 Will change.H 3 E M E S D, 41.11, CR Delete “may be” fro
flaws and defective. Find the probability that the part is defective given it has surface flaws. And the probability that the part is defective given it doesn’t have surface flaws. Surface flaws (F) No surface Flaws (F′) Defective (D
Excerpt from the control flaws table associated to the human driver controller 58 control flaws for the Examine the human driver controller interactions of direct controllers Identify control flaws: Perception (feedback) Mental Models Decision-making Action Execution 48 control flaws
5 D 3 4 E Step 5 looks time critical, establish criteria for maximum time to trip CB-1 1/1 6 D 3 2 S Need a pause between response to first rod drift and second alarm, to discuss actions required in
Complete Dragon Magazine 3.5 Feats, Flaws, & Styles Introduction 1 INTRODUCTION Collected here is a listing of nearly all Feats, Flaws, and Fighting Styles 3.5 described in DRAGON MAGAZINE issues 309-360 and Dungeon Magazine issues 100-154. Omitted stats were done so only because they appeared in other books or
3.1 Test Plate #1 20 3.2 Test Plate //5 20 3.3 Test Plates #10 and #12 20 3.4 General Length Observations - All Flaws 21 4. Observations on Vertical Extent Measurement Techniques 25 d) Small flaws 25 Cii) Large Flaws 27 5. Observations on Sizing Repeatability 2B 5.1 Decibel Drop Methods - Repeatability 28
similarity of operating systems and their security problems, specific security flaws are formally classified according to a taxonomy developed here. This classification leads to a clearer understanding of security flaws and aids in analyzing new systems. The discussions of security flaws and the security
psychometric properties of the Beck Depression Inventory for Youth (BDI-Y) in a sample of 859 girls that were ages 9-13 years by age level and for selected races or ethnic groups. While research on the psychometric properties of the BDI-Y has been scarce the suggestion of its soundness has been made. In term of standardization, the manual provides separate normative information for 7 to 10 .
Academic writing styles can vary from journal to journal, so you have to check each publication’s guide for writers and follow it carefully and/ or copy other papers in it. Academic writing titles cultural differences and useful phrases Academic papers often have a title with two parts. If the title of an academic paper has two parts, the two parts are usually separated by a colon .
