Kiến trúc máy tính và hợp ngữ - Chapter 5: Requirement validation

Requirements Validation Check that the right product is being built Refers to the set of activities that ensure that software correctly implements a specific function Requirements Verification Check that product is being built right refers to a different set of activities that ensure that the software that has been built is traceable to customer requirements 2 Requirements Verification and Validation Techniques Simple checks Prototyping Functional test design User manual development Reviews and inspections Is a whole life-cycle process - V & V must be applied at each stage in the software process. Has two principal objectives The discovery of defects in a system; The assessment of whether or not the system is useful and useable in an operational situation. 4  Help ensure delivery of what the client wants  Need to be performed at every stage during the (requirements) process  Elicitation Checking back with the elicitation sources  Analysis Checking that the domain description and requirements are correct  Specification Checking that the defined system requirement will meet the user requirements under the assumptions of the domain/environment Checking conformity to well-formedness rules, standards Check that the right product is being built Ensures that the software being developed (or changed) will satisfy its stakeholders Checks the software requirements specification against stakeholders goals and requirements 11/7/2014 6 Check that product is being built right Ensures that each step followed in the process of building the software yields the right products Checks consistency of the software requirements specification artefacts and other software development products (design, implementation, ...) against the specification 11/7/2014 7 8 Simple checks Prototyping Functional test design User manual development Reviews and inspections Model-Based V&V Verify that the requirements are well written (according to the criteria already discussed) Various checks can be done using traceability techniques Given the requirements document, verify that all elicitation notes are covered Tracing between different levels of requirements Checking goals against tasks, features, requirements Involves developing a traceability matrix Ensures that requirements have been taken into consideration (if not there should be a reason) Ensures that everything in the specification is justified Excellent for validation by users and customers More accessible than specification Demonstrate the requirements and help stakeholders discover problems Come in all different shapes and sizes From paper prototype of a computerized system to formal executable models/specifications Horizontal, vertical Evolutive, throwaway Important to choose scenarios or use cases for elicitation session Prototyping-based validation steps Choose prototype testers Develop test scenarios Careful planning is required to draw up a set of test scenarios which provide broad coverage of the requirements Users should not just play around with the system as this may never exercise critical system features Execute test scenarios Document problems using a problem reporting tool Same reasoning as for functional test design Has to be done at some point Reveals problems earlier Forces a detailed look at requirements Particularly useful if the application is rich in user interfaces / for usability requirements Typical information in a user manual Description of the functionality How to get out of trouble How to install and get started with the system A group of people read and analyze requirements, look for potential problems, meet to discuss the problems, and agree on a list of action items needed to address these problems A widely used requirements validation technique Lots of evidence of effectiveness of the technique Can be expensive Careful planning and preparation Pre-review checking Need appropriate checklists (must be developed if necessary and maintained) 14  Different types of reviews with varying degrees of formality exist (similar to JAD vs. brainstorming sessions) Reading the document A person other than the author of the document Reading and approval (sign-off) Encourages the reader to be more careful (and responsible) Walkthroughs Informal, often high-level overview Can be led by author/expert to educate others on his/her work Formal inspections Very structured and detailed review, defined roles for participants, preparation is needed, exit conditions are defined E.g., Fagan Inspection Different types of reviews Focused inspections Reviewers have roles, each reviewer looks only for specific types of errors Active reviews Author asks reviewer questions which can only be answered with the help of the document to be reviewed Plan review The review team is selected and a time and place for the review meeting is chosen Distribute documents The requirements document is distributed to the review team members Prepare for review Individual reviewers read the requirements to find conflicts, omissions, inconsistencies, deviations from standards, and other problems Hold review meeting Individual comments and problems are discussed and a set of action items to address the problems is established  Follow-up actions  The chair of the review checks that the agreed action items have been carried out  Revise document  Requirements document is revised to reflect the agreed action items  At this stage, it may be accepted or it may be re-reviewed • Reviews should involve a number of stakeholders drawn from different backgrounds – People from different backgrounds bring different skills and knowledge to the review – Stakeholders feel involved in the RE process and develop an understanding of the needs of other stakeholders – Review team should always involve at least a domain expert and a user  Requirements clarification The requirement may be badly expressed or may have accidentally omitted information which has been collected during requirements elicitation  Missing information Some information is missing from the requirements document  Requirements conflict There is a significant conflict between requirements The stakeholders involved must negotiate to resolve the conflict  Unrealistic requirement The requirement does not appear to be implementable with the technology available or given other constraints on the system Stakeholders must be consulted to decide how to make the requirement more realistic Reviews can be expensive because they involve many people over several hours reading and checking the requirements document We can reduce this cost by asking someone to make a first pass called the pre-review Check the document and look for straightforward problems such as missing requirements (sections), lack of conformance to standards, typographical errors, etc. Reviewer is asked to use the specification Author poses questions for the reviewer to answer that can be answered only by reading the document Author may also ask reviewer to simulate a set of scenarios Essential tool for an effective review process List common problem areas and guide reviewers May include questions an several quality aspects of the document: comprehensibility, redundancy, completeness, ambiguity, consistency, organization, standards compliance, traceability ... There are general checklists and checklists for particular modeling and specification languages Checklists are supposed to be developed and maintained Functional tests at the system level must be developed sooner or later... Designing these tests may reveal errors in the specification (even before designing and building the system) Some software development processes (e.g., agile methods) begin with tests before programming  Test-Driven Development (TDD) Defect testing Tests designed to discover system defects. A successful defect test is one which reveals the presence of defects in a system. Validation testing Intended to show that the software meets its requirements. A successful test is one that shows that a requirements has been properly implemented. Test Planing: Define a software test plan by specifying: a test schedule for a test process, test requirements and items, test strategy and supporting tools Test Design and Specification Conduct software design based well-defined test generation methods. Specify test cases to achieve a targeted test coverage. Test Set up: Testing Tools and Environment Set-up Test Operation and Execution Run test cases manually or automatically Test data: Inputs which have been devised to test the system Test cases: Inputs to test the system and the predicted outputs from these inputs if the system operates according to its specification Example: Test case for Withdrawals on an ATM Validate that a withdrawal of a multiple of $20, between $20-$300 can be done Inp ut No Functional input Expect ed result Actual result Passed/ Failed 1 Enter a withdrawal of a multiple of $20, between $20-$300 Valid 2 Enter withdraw amount <20$ Invalid 3 Enter withdraw amount >300$ Invalid 4 Enter a withdrawal of non-$20 multiples between $20-$300 Invalid 5 Validate that a valid withdrawal amount must be below the account balance Valid 6 Validate that the withdrawal received is equal to the amount requested Valid