Process Modeling and Simulation Experiments for Software Engineering

Please download to get full document.

View again

of 21
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

Documents

Published:

Views: 0 | Pages: 21

Extension: PPT | Download: 0

Share
Related documents
Description
Process Modeling and Simulation Experiments for Software Engineering. Nancy S. Eickelmann,PhD ... Process simulation experiments ...
Tags
Transcript
Title: Process Modeling and Simulation Experiments for Software Engineering 1 Process Modeling and Simulation Experiments for Software Engineering Nancy S. Eickelmann,PhDMotorola Labs1303 E. Algonquin Rd.Annex-2Schaumburg, IL 60196Phone (847) 310-0785Fax (847) 576-3280Nancy.Eickelman n_at_motorola.com USC-CSE Octoberr 23-26, 2001 Dr. Nancy Eickelmann 2 Overview
  • Process Modeling and Simulation
  • Who Uses It CMM High Maturity Organizations
  • How to use it for Defect Prevention
  • Simulation Experiments for Software Engineering
  • Internal Validity
  • External Validity
  • Design of Experiments
  • 3 Process Modeling and Simulation for High Maturity Organizations Software Lifecycle Process Process Performance Cost, Quality, Schedule Project Data Process and Product 4 State of the Practice Increasing Process Maturity Source SEI Web Site SEMA Report for March 2000 5 Level 5 KPAs Optimizing
  • Defect Prevention
  • Goal 1- Defect prevention activities are planned
  • Goal 2- Common causes of defects are identified
  • Goal 3- Common causes of defects are prioritized and eliminated
  • Technology Change Management
  • Goal 1- Incorporation of technology changes are planned
  • Goal 2- New technologies are evaluated to determine their effect on quality and productivity
  • Goal 3- Appropriate new technologies are transferred into practice
  • Process Change Management
  • Goal 1- Continuous process improvement CPI is planned
  • Goal 2- Organization wide process improvement
  • Goal 3- Standard processes are improved continuously
  • 6 Defect Prevention
  • Defect prevention is defined as an activity of continuous institutionalized learning during which common causes of errors in work products are systematically identified and process changes eliminating those causes are made.
  • 7 What is Required for Defect Prevention?
  • A measurement program that provides full lifecycle in-process visibility
  • Knowledge of how and when defects by type, severity, and impact are introduced into the product
  • Methods to improve the process that will result in defect prevention
  • 8 From a Risk Management Perspective
  • Defect prevention through risk management means engaging in a set of planning, controlling, and measuring activities that result in obviating, mitigating or ameliorating defect causing conditions.
  • 9 Process Simulation Models
  • Experimental Simulation
  • Qualitative and quantitative results based on non-deterministic or hybrid simulation model
  • mirrors a segment of the real world
  • control of variables is high
  • supports testing of causal hypothesis
  • results can be replicated
  • high internal validity
  • high external validity, generalizability
  • 10 Key Issues for Empirical Studies
  • First, software engineering has a large number of key variables that have different degrees of significance depending on the process lifecycle, organizational maturity, degree of process automation, level of expertise in the domain, computational constraints on the product, required properties of the product.
  • Second, the individual key variables required to mirror the real world context have the potential property of extreme variance in the set of known values within the same context or across multiple contexts. For instance, programmer productivity a key variable in most empirical studies has been documented at 101 and 251 variances in the same context.
  • Third, software engineering domain variables, in combination, may create a critical mass or contextual threshold not present when studied in isolation. To identify variables that co-vary and have interdependent relationships statistical methods are applied to the data sets.
  • 1986 IEEE TSE, Basili, Selby and Hutchins
  • 11 Empirical Research Summary
  • Experimental Simulation
  • Qualitative and quantitative results based on non-deterministic or hybrid simulation model
  • Math Modeling quantitative results based on a deterministic model
  • Mirrors a segment of the real world, control of variables is high, supports testing of causal hypothesis, results can be replicated, high internal validity and generalizability
  • Captures real world context in which to isolate and control variables
  • Researcher bias can be introduced through selection of variables, parameters and assumptions concerning the model. Modeling requires high degree of analytical skill, and interdisciplinary knowledge
  • Results are not typically generalizable to other populations or environmental contexts, researcher bias is common.
  • 12 Factors Jeopardizing Research Internal Validity
  • History - events occurring between the 1st and 2nd measurement of the experimental variables
  • Maturation - processes impacting study results pertaining to the passage of time, i.e., growing tired, growing hungry, growing older, undocumented reliability growth or decay
  • Testing - the effects of taking a test upon the scores of the 2nd test
  • Instrumentation - changes in the measuring instrument, changes in the observers or record keeper perceptions
  • Statistical regression - group selection based on extreme scores
  • Bias - differential selection of comparison groups
  • Experimental mortality - loss of respondents
  • Selection/Maturation interaction - confounding variable mistaken for dependent variable
  • 13 Factors Jeopardizing Research External Validity (Generalizability)
  • Testing interaction or reactive effects - altered respondent sensitivity due to pre-test measurement
  • Interaction effects - confounding effects from selection bias and experimental variable
  • Reactive effects of experimental arrangements - obviates applicability of results to persons or contexts not exposed under the experimental setting
  • Multiple treatment interference - occurs when the respondent pool is reused repeatedly
  • 14 How We Assure Internal Validity Solomon Four Group Design 15 How We Assure Internal Validity X X,Y,Z M3 X,Y M4 X,Z Simulation Experiment Design 16 Initialization Sub-Module
  • Set the initial parameters for the model
  • Inputs
  • Initial Defects X
  • Detection Effectiveness Y
  • Correction Effectiveness
  • Number of Inspections
  • Inspection Size Z
  • Delta Size
  • Resources (Moderator, Author, Librarian, Recorder, Inspector, Reader, Other)
  • Output
  • Item Out
  • 17 Fagan Inspection Sub-Module
  • Calculate duration and number of defects found and removed
  • Inputs
  • InspectedItem
  • OverviewIn, ThirdHourIn
  • Output
  • FaganInspectionDurationOut
  • OverviewDurationOut
  • PlanningDurationOut
  • PreparationDurationOut
  • InspectionDurationOut
  • ThirdHourDurationOut
  • ReworkDurationOut
  • FollowUpDurationOut
  • MinorDefectFoundOut
  • MajorDefectFoundOut
  • DefectRemovalOut
  • ItemOut
  • 18 Preliminary Results
  • Captures Numeric Graphical Simulation Results
  • Inputs
  • Selected Intermediate and Final Module Values
  • Outputs
  • Duration for Each Activity
  • Number of Major Defects Found
  • Number of Minor Defects Found
  • Number of Defects Removed
  • Minimum Maximum Number of Days Expended
  • 19 What We Need for Empirical Studies in the Software Engineering Domain
  • Process simulation experiments
  • Capture and replicate the variables of the real world environment
  • variable variances are isolated and documented
  • variables are studied in isolation or in combination to isolate and document critical mass effects
  • the cost to replicate the multiple real world environments and evaluate across projects and organizations is much less than field studies, longitudinal case studies or controlled experiments
  • we can replicate other empirical studies and evaluate applicability and generalizability of results
  • 20
  • Thank You!
  • Nancy S. Eickelmann,PhDMotorola Labs1303 E. Algonquin Rd.Annex-2Schaumburg, IL 60196Phone (847) 310-0785Fax (847) 576-3280Nancy.Eickelman n_at_motorola.com
  • Recommended
    View more...
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks