QUALITATIVE DATA ANALYSIS

QUALITATIVE DATA ANALYSIS

INSTRUCTIONAL GOAL Using qualitative data analysis techniques, analyze a provided data set and reach appropriate conclusions.

INSTRUCTIONAL OBJECTIVES Describe the purpose and use of qualitative data analysis. Explain how coding is used to condense data. Describe the different types of coding methodologies used in data condensation. Describe the purpose and scope of displaying data in qualitative data analysis.

INSTRUCTIONAL OBJECTIVES Describe different matrices and data displays used to explore, order, and explain qualitative data analysis results. Describe the tactics used to generate meaning from qualitative data sets. Describe the tactics used to test or confirm findings. Using appropriate qualitative data analysis techniques, analyze a provided data set and reach appropriate conclusions.

QUALITATIVE ANALYSIS Used to analyze collected data and form performance conclusions Used to analyze multiple data sets – Identify systemic issues – Determine extent of condition – Determine corrective action effectiveness

QUALITATIVE ANALYSIS More subjective, less restrictive than Quantitative Analysis Often done unconsciously when we compare/contrast It is based on finding patterns or recurring themes and deviations to those patterns Litmus test – would a reasonable person reach the same conclusion?

QUALITATIVE ANALYSIS Three concurrent flows of activity: Data Condensation Data Display Forming and Verifying Conclusions

DATA CONDENSATION AND CODING Codes are assigned to data chunks to detect recurring patterns Similar codes are clustered together Interrelationships are constructed to develop higher-level analytic meanings

DATA CONDENSATION – CODES & CODING I think they are taking skills back to their organizations and making improvements, such as better tooling, unpacking and characterization processes for their waste handling. I don’t really have too much interaction with them outside of the productivity improvement initiative, but I think that the operators may be inclined to leverage some of the other topical improvements being discussed by other organizations. I believe that is true because of the questions and interest the operators express when other topics are discussed. The operators became more engaged in the conversations and were providing some good insight. During conversations with the operators they were taking ideas discussed in the productivity initiative meeting and applying them to their own work sites, such as better equipment organization and videotaping tasks that are not routinely performed for refresher training. Learning Improvement s Communication

CODING METHODOLOGIES Creating codes – Develop a “start list” (Deductive) 12-50 codes – May need to add codes (Inductive) Revising codes –Codes will change Some don’t work – delete Others work too well – need subcodes Software applications (QDA Miner, Qiqqa, TAMS, ect.)

CODING METHODOLOGIES Related Defined PRIMARY CODES CF2.1 CF2.2 CF2.3 CF3.1 CF3.2 CF3.3 CF3.4 CF3.5 ISM CORE FUNCTIONS 2&3 Analyze hazards LTA Not all hazards identified Wrong hazards identified Identif. & Implem. of controls LTA All controls not identified All controls not implemented Wrong controls identified Wrong controls implemented SUBCODES PROCESS HUMAN TRAINING ERROR ERROR LTA (A) (B) (C) CF2.1A CF2.1B CF2.1C CF2.2A CF2.2B CF2.2C CF2.3A CF2.3B CF2.3B CF3.1A CF3.1B CF3.1C CF3.2A CF3.2B CF3.2C CF3.3A CF3.3B CF3.3C CF3.4A CF3.4B CF3.4C CF3.5A CF3.5B CF3.5C SYSTEM ERROR (D) CF2.1D CF2.2D CF2.3D CF3.1D CF3.2D CF3.3D CF3.4D CF3.5D Assign to narrative phrases, blocks or paragraphs

VALIDATING YOUR CODES ?*! ?*! ?*! ?*! ?*! ?*! ?*! ?*!

DATA DISPLAYS Display A visual format that presents information systematically Used in qualitative analysis matrices Display contingent on what you are trying to understand

DATA DISPLAY Preparing the Display Reviewing the Display Use multiple data displays Start with a quick scan Add qualitative explanations to quantitative displays Reach explanatory conclusion Document analytical work, it must validate the conclusion “Trust, but Verify” – check exploratory data

MATRIX BUILDING CONSIDERATIONS Look at your question(s) and key variables Get a peer review Try to make the display one "page" (screen or printed) Don’t try to include 12 variables in rows or columns The simplest matrices are two dimensions

Side-by-Side Display - Evidence Comparison Line of Inquiry Document Review C00-1.1 Walk downs are performed for procedure/process changes SOP-XXX, Section XX, requires walk downs be performed by engineers and operators for Level I/II procedure/process changes Interview Four of five operators interviewed indicated that although Level I/II procedure changes had been discussed in Pre-Job briefings, they were not routinely involved in walk downs. One Operator relayed that they had been involved in one walk down during the last 9 months. The individual did not recall signing or initialing the required XX.X form. Six of the eight process engineers reported that they did not routinely involve operating staff in walk downs because of time and resource complaints. Observations Observed walk down of SOP-XX.X Level II Operating Procedure. Changes were highlighted as required; however there were no Operators present. Personnel present (CSE, Engineering, Operations Supervisor)

Matrix Example Participant # A/V Quality Presentation Agenda/Objectives Other

Data driven by matrix columns and rows DISPLAY CONCLUSIONS

FORMING AND VERIFYING CONCLUSIONS Note patterns, explanations, causal flows – use tactics Verify the conclusion – use tests

TACTICS FOR IDENTIFYING LEADS, TRENDS, AND CODES See Tab

TACTICS FOR TESTING OR CONFIRMING CONCLUSIONS See Tabs

EXERCISE PREP Scenario Tab Master Data Set Tab Query Data Sets and Displays Tab Forming and Verifying Conclusion Template Tab

EXERCISE – QUALITATIVE DATA ANALYSIS

EXERCISE INSTRUCTIONS 1. Using the Files provided: Scenario Master Data Set Query Data Sets and Displays Tactical Checklists 2. Perform a QDA (as a team in the breakout room assigned) 3. Document results on the Forming and Verifying Conclusion template 4. Save File and Share in Open Session

SHARE RESULTS

EXERCISE FORMING CONCLUSIONS

REVIEW QDA is used to analyze collected oversight data to form fact-based conclusions and decisions Analyze multiple data sets to identify systemic issues. Verify conclusions

REVIEW Three concurrent flows of activity Data Condensation Data Display Forming and Verifying Conclusions

REVIEW Data Condensation - selecting, simplifying, abstracting, and/or transforming data I think they are taking skills back to their organizations and making improvements, such as better tooling, unpacking and characterization processes for their waste handling. I don’t really have too much interaction with them outside of the productivity improvement initiative, but I think that the operators may be inclined to leverage some of the other topical improvements being discussed by other organizations. I believe that is true because of the questions and interest the operators express when other topics are discussed. The operators became more engaged in the conversations and were providing some good insight. During conversations with the operators they were taking ideas discussed in the productivity initiative meeting and applying them to their own work sites, such as better equipment organization and videotaping tasks that are not routinely performed for refresher training. Learning Improvement s Communication

REVIEW Data driven by matrix columns and rows Data Display - an organized, compressed assembly of information

REVIEW Forming and Verifying Conclusions Note patterns, explanations, causal flows – use tactics Verify the conclusion – use tests

REVIEW Let the Methods Support Each Other QUALITATIVE ANALYSIS QUANTITATIVE ANALYSIS Let your quantitative and your qualitative analysis “work together" Ask - Do the patterns corroborate the findings of any corresponding analyses?

Questions or Comments