# Infosys Whitepaper
Title: Test automation framework – how to choose the right one for digital transformation?
Author: Infosys Limited
Format: PDF 1.7

---

  Page: 1 / 4

---

 VIEW POINT TEST AUTOMATION FRAMEWORK –  HOW TO CHOOSE THE RIGHT ONE FOR  DIGITAL TRANSFORMATION?

---

  Page: 2 / 4

---

 What is a test automation  framework and what are its  different types? A test automation framework is a combin-  ation of guidelines, coding standards,  concepts, practices, processes, project  hierarchies, reporting mechanism, test  data, to support automation testing.  A tester follows these guidelines  while automating applications to take  advantage of various productive results.<|endoftext|>Introduction Digitalization and the disruption caused by the adoption of digital technologies are rapidly changing the world. Speed matters a lot in all  IT operations, and this calls for a paradigm shift in quality assurance (QA). Quality at high speed is the key focus in digital assurance, and  organizations want to deliver quality products much faster than ever before. This is making QA teams to bank on test automation. From the  initial automation of regression tests, the industry is moving towards progressive automation and day one automation. At the same time,  extreme automation and zero touch automation are the buzz words in the QA world these days.  Various advancements have evolved in the area of automation testing. However, it is critical that organizations choose the right automation  framework, which is considered a critical factor for its success. In this document, we will explore the different types of automation  frameworks, and how to choose the right framework which will help in achieving the digital assurance goals of the organizations.<|endoftext|>There are many types of test automation  frameworks available in the market, and  the most popular ones are listed here.  Each one of these frameworks has their  individual characteristics and features.<|endoftext|>Let us now examine some of the popular  frameworks and understand their pros,  cons, and usability recommendations: Linear Functional Decomposition /Modular Data Driven Keyword Driven Hybrid BDD Automation Framework Types Keyword-driven framework In the keyword-driven framework, testers  create various keywords and associate  different actions or functions with each  of these keywords. Function library  contains the logic to read the keywords  and call and perform the associated  actions. Generally, test scenarios are  written in excel sheets. The driver script  reads the scenario and performs test  execution. This is used in situations  where the testers who create test scripts  the strengths of the different frameworks  and mitigates their weaknesses. It  is highly robust, flexible, and more  maintainable. However, this requires  strong technical expertise to design and  maintain.<|endoftext|>  Behavior-driven development  framework Behavior-driven development (BDD)  framework automates validations in  an easily readable and understandable  format to business analysts, developers,  testers, etc. Such frameworks do not  necessarily require the user to be  acquainted with any programming  language. There are different tools  available for BDD like Cucumber, JBehave,  and more which work along with other  test automation tools.  This framework is  more suitable for applications using agile  methodology and where user stories and  early automation are required. It focuses  on the behavior of the system rather than  the implementation aspect of the system.   The traceability between requirements  and scripts is maintained throughout, and  test scripts are easy to understand for the  business users.<|endoftext|>Pillars of the right framework  for the digital era Automation can improve quality and  lead to higher testing efficiency. Hence,  it is important to plan it well and make  the right choice of tools and frameworks.  When test automation uses the right  framework based on the context, it  yields great benefits. Hence, it is worth  understanding the key requirements of  the framework, before choosing the   right one.<|endoftext|>have less programming expertise,  whereas framework creation is done by  automation experts Hybrid framework The hybrid automation framework is  created by combining distinct features of  two or more frameworks. This enhances  External Document © 2018 Infosys Limited

---

  Page: 3 / 4

---

 Some key aspects of automation  framework to look for during the digital  assurance journey are provided below: Extreme automation Digital transformation programs, big  data, cloud, and mobility are changing  the way testing is being done. Leaders  in testing are moving towards extreme  automation to achieve a faster time  to market. Extreme automation is the  key, and automating every part of the  testing process instead of just regression  is crucial now. A framework which is  more scalable and facilitates lifecycle  automation as well as broader test  coverage is needed for digital assurance  programs.<|endoftext|>Technology and tool agnostic  approach The landscape of tools in QA is becoming  wider day by day. There are too many  tools and frameworks, which poses a lot  of integration challenges. Hence, it is  imperative to choose a framework which  is technology and tool agnostic and  supports various tools and technologies.  The framework needs to address  enterprise-level automation strategy and  goals instead of catering to just a single project goals.<|endoftext|>Script less capabilities Automate the automation, and look out  for scriptless automation avenues.  Most  software testers and business users find  it challenging to learn programming  languages such as Java, Visual Basic,  etc. well enough to write the scripts  that the test automation demands.  There are frameworks and accelerators  available with user-friendly graphical  user interfaces (GUI) which help to create  automation scripts in a much easier way  than having to know and write code in  any specific programming language.  Choosing a framework which helps to  create a test script from the recorded  script or based on the input from a  spreadsheet will help in accelerating  automation and reduce dependency on  skilled resources.<|endoftext|>True shift left attitude Digitalization and frequent releases call  for day one automation. Gone are the  days when the automation team would  wait until the application is built and  start automation activities thereafter.  The need of the hour is to shift extreme  left and start automation during the  requirement gathering phase of the  systems development life cycle (SDLC)  itself. Automation framework with  exhaustive reusable library and support  for BDD will help both business users  and QA teams to start automation  activities early in the life cycle.<|endoftext|>Omnichannel, mobility, and  cloud features Organizations today are focusing more  on digital assurance, but it is important  to test real user behaviors and to test on  multiple devices such as various mobiles,  tablets, platforms, and such. Hence,  the chosen test automation framework  needs to facilitate testing on multiple  devices to ensure a uniform experience  across devices. If the framework supports  the reuse of the script used for online  or desktop testing for mobile testing as  well with minimal rework, it will help in  saving much effort. When addressing the  multifaceted needs of mobile testing,  conducting comprehensive testing  across hundreds of different devices,  brands, models, and different operating  system combinations is tedious. A  framework that facilitates integration  with cloud infrastructure will be an  added advantage.<|endoftext|>External Document © 2018 Infosys Limited

---

  Page: 4 / 4

---

 Zero touch automation As DevOps is slowly taking over the  IT landscape, it is vital to reduce the  distance between development and  deployment. Test scripts need to be  executed in an unattended manner  without requiring much manual  intervention. Remote execution, parallel  execution, zero touch execution, and  execution from continuous integration  tools like Jenkins and Hudson, when  supported by automation framework, will  help a lot in managing multiple sprints  and shorter cycles better.  Seamless integration With a plethora of tools being used in  application development and testing  landscape, it is important that the  automation tool and the framework  chosen facilitate integration with various  tools.  Hence, it is imperative that the  chosen automation framework and tool  integrate with test management tools,  defect tracking tools, build tools, analytics  tools, and continuous integration tools in  the landscape.<|endoftext|>User-friendly reporting Agile and Dev Ops has brought the  business, development and QA teams  to work together. The ability to run a  high volume of tests is of little use if  the results of the tests are not easy to  understand by various stakeholders  involved. The framework has to facilitate  automatic generation of reports of the  test execution and show the results in  an easy-to-read format. Though most  of the market tools give few reporting  options, they are not self-explanatory  and adequate. Hence, the framework  with good reporting capabilities such as  HTML reports, live execution dashboard,  screen shots in case of failures, and video  reporting of the execution options will  be very helpful. Automation framework  facilitating detailed test result reporting  reduces the overall effort to a greater  extent.<|endoftext|>About the author Indumathi Devi, a project manager with Infosys, has 13+ years of experience in software testing. She has effectively executed a multitude  of automation projects and designed and developed automation frameworks. Using her strong working knowledge of multiple test


Continue # Infosys Whitepaper
  automation tools, including open source and commercial ones, Indu has worked with numerous clients in implementing robust test  automation solutions.<|endoftext|>Conclusion  No one size fits all. This perfectly  holds true when it comes to  framework selection. Since every  project is unique, the challenges,  duration, and tools choices may vary.  Organizations seeking agility in their  business processes need to onboard  robust test automation solutions that  ensure superior software quality.  Successful test automation  frameworks for digital assurance are  the ones which support extreme  automation, omnichannel testing,  zero touch execution of test  scripts, and have some or all of the  key aspects detailed above. We  recommend that organizations select  an automation framework that can  lead to smarter automation, better  overall results, productivity benefits,  and cost efficiencies in the highly  dynamic digital landscape.<|endoftext|>© 2018 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys  acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this  documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the  prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.  For more information, contact askus@infosys.com Infosys.com | NYSE: INFY Stay Connected

***


# Infosys Whitepaper
Title: The right testing strategy for AI systems - An Infosys viewpoint
Author: Infosys Limited
Format: PDF 1.6

---

  Page: 1 / 8

---

 PERSPECTIVE THE RIGHT TESTING STRATEGY FOR AI  SYSTEMS AN INFOSYS VIEWPOINT VENKATESH IYENGAR, AVP - Group Practice Engagement Manager, Infosys SUNDARESA SUBRAMANIAN G, Practice Engagement Manager, Infosys

---

  Page: 2 / 8

---

 External Document © 2018 Infosys Limited Abstract Over the years, organizations have invested significantly  in optimizing their testing processes to ensure continuous  releases of high-quality software. When it comes to artificial  intelligence, however, testing is more challenging owing to  the complexity of AI. Thus, organizations need a different  approach to test their AI frameworks and systems to ensure  that these meet the desired goals. This paper examines  some key failure points in AI frameworks. It also outlines  how these failures can be avoided using four main use cases  that are critical to ensuring a well-functioning AI system.<|endoftext|>The hierarchy of evolution of AI Introduction Experts in nearly every field are in a race to discover how to replicate brain functions – wholly or partially. In fact, by 2025, the value of the  artificial intelligence (AI) market will surpass US $100 billion1. For corporate organizations, investments in AI are made with the goal of  amplifying the human potential, improving efficiency and optimizing processes. However, it is important to be aware that AI too is prone to error  owing to its complexity. Let us first understand what makes AI systems different from traditional software systems: S.No Software systems AI systems 1 Features – Software is deterministic, i.e., it is pre-pro- grammed to provide a specific output based on a given set  of inputs Features – Artificial intelligence/machine learning (AI/ML) is  non-deterministic, i.e., the algorithm can behave differently for  different runs 2 Accuracy – Accurate software depends on the skill of the  programmer and is deemed successful if it produces an  output in accordance with its design Accuracy – Accuracy of AI algorithms depends on the training  set and data inputs 3 Programming – All software functions are designed based  on if-then and for loops to convert input data to output  data Programming – Different input and output combinations are fed  to the machine based on which it learns and defines the function 4 Errors – When software encounters an error, remediation  depends on human intelligence or a coded exit function Errors – AI systems have self-healing capabilities whereby they  resume operations after handling exceptions/errors  Fig 1: Hierarchy of AI’s evolution and techniques Fig 1: Hierarchy of AI’s evolution and techniques                                              Visualization  Machine learning and  analytics  Input data conditioning  Learning process – data sources  Feedback  Feedback – From sensors, devices, apps, and systems  Visualization – Custom apps, connected devices, web, and bots  Machine learning and analytics – Cognitive learning/algorithms  Input data conditioning – Big data stores and data lakes  Data sources – Dynamic or static sources like text, image,  speech, sensor, video, and touch

---

  Page: 3 / 8

---

 External Document © 2018 Infosys Limited External Document © 2018 Infosys Limited The above figure shows the sequential stages of AI algorithms. While each stage is necessary for successful AI programs, there are some typical  failure points that exist within each stage. These must be carefully identified using the right testing technique as shown in the table below: S.No Evolution stage in AI Typical failure points How can they be detected in testing 1 Data sources –  Dynamic or static  sources • Issues of correctness, completeness and appropriateness  of source data quality and formatting • Variety and velocity of dynamic data resulting in errors • Heterogeneous data sources  • Automated data quality checks  • Ability to handle heterogeneous data during  comparison  • Data transformation testing  • Sampling and aggregate strategies 2 Input data condition- ing – Big data stores  and data lakes • Incorrect data load rules and data duplicates • Data nodes partition failure • Truncated data and data drops  • Data ingestion testing  • Knowledge of development model and  codes  • Understanding data needed for testing  • Ability to subset and create test data sets 3 ML and analytics –  Cognitive learning/ algorithms • Determining how data is split for training and testing • Out-of-sample errors like new behavior in previously  unseen data sets • Failure to understand data relationships between enti- ties and tables • Algorithm testing  • System testing  • Regression testing  4 Visualization – Cus- tom apps, connected  devices, web, and  bots • Incorrectly coded rules in custom applications resulting  in data issues •
Formatting and data reconciliation issues between  reports and the back-end • Communication failure in middleware systems/APIs  resulting in disconnected data communication and  visualization • API testing  • End-to-end functional testing and automa- tion  • Testing of analytical models  • Reconciliation with development models 5 Feedback – From sen- sors, devices, apps,  and systems • Incorrectly coded rules in custom applications resulting  in data issues • Propagation of false positives at the feedback stage  resulting in incorrect predictions • Optical character recognition (OCR) testing  • Speech, image and natural language pro- cessing (NLP) testing  • RPA testing  • Chatbot testing frameworks The right testing strategy for  AI systems Given the fact that there are several failure  points, the test strategy for any AI system  must be carefully structured to mitigate  risk of failure. To begin with, organizations  must first understand the various stages  in an AI framework as shown in Fig 1. With  this understanding, they will be able to  define a comprehensive test strategy with  specific testing techniques across the entire  framework. Here are four key AI use cases  that must be tested to ensure proper AI  system functioning: • Testing standalone cognitive features  such as natural language processing  (NLP), speech recognition, image  recognition, and optical character  recognition (OCR) • Testing AI platforms such as IBM  Watson, Infosys NIA, Azure Machine  Learning Studio, Microsoft Oxford,  and Google DeepMind • Testing ML-based analytical models • Testing AI-powered solutions such as  virtual assistants and robotic process  automation (RPA)

---

  Page: 4 / 8

---

 External Document © 2018 Infosys Limited Use case 1: Testing standalone cognitive features Natural language processing  (NLP) • Test for ‘precision’ return of the keyword, i.e., a fraction of relevant instances among the total retrieved  instances of NLP • Test for ‘recall’, i.e., a fraction of retrieved instances over total number of retrieved instances available • Test for true positives (TPs), true negatives (TNs), false positives (FPs), and false negatives (FNs). Ensure  that FPs and FNs are within the defined error/fallout range Speech recognition inputs • Conduct basic testing of the speech recognition software to see if the system recognizes speech inputs  • Test for pattern recognition to determine if the system can identify when a unique phrase is repeated  several times in a known accent and whether it can identify the same phrase when it is repeated in a  different accent • Test deep learning, the ability to differentiate between ‘New York’ and ‘Newark’ • Test how speech translates to response. For example, a query of “Find me a place I can drink coffee”  should not generate a response with coffee shops and driving directions. Instead, it should point to a  public place or park where one can enjoy his/her coffee Image recognition • Test the image recognition algorithm through basic forms and features • Test supervised learning by distorting or blurring the image to determine the extent of recognition by  the algorithm • Test pattern recognition by replacing cartoons with the real image like showing a real dog instead of a  cartoon dog • Test deep learning using scenarios to see if the system can find a portion of an object in a larger image  canvas and complete a specific action Optical character recognition • Test OCR and optical word recognition (OWR) basics by using character or word inputs for the system to  recognize • Test supervised learning to see if the system can recognize characters or words from printed, written or  cursive scripts • Test deep learning, i.e., whether the system can recognize characters or words from skewed, speckled  or binarized (when color is converted to grayscale) documents • Test constrained outputs by introducing a new word in a document that already has a defined lexicon  with permitted words  Use case 2: Testing AI platforms Testing any platform that hosts an AI


Continue # Infosys Whitepaper
 framework is complex. Typically, it follows many of the steps used during functional testing.  Data source and conditioning testing • Verify the quality of data from various systems – data correctness, completeness and appropriateness along with format checks, data  lineage checks and pattern analysis • Verify transformation rules and logic applied on raw data to get the desired output format. The testing methodology/automation  framework should function irrespective of the nature of data – tables, flat files or big data • Verify that the output queries or programs provide the intended data output • Test for positive and negative scenarios Algorithm testing • Split input data for learning and for the algorithm • If the algorithm uses ambiguous datasets, i.e., the output for a single input is not known, the software should be tested by feeding a set  of inputs and checking if the output is related. Such relationships must be soundly established to ensure that algorithms do not have  defects • Check the cumulative accuracy of hits (TPs and TNs) over misses (FPs and FNs) API integration • Verify input request and response from each application programming interface (API) • Verify request response pairs • Test communication between components – input and response returned as well as response format and correctness  • Conduct integration testing of API and algorithms and verify reconciliation/visualization of output System/regression testing • Conduct end-to-end implementation testing for specific use cases, i.e., provide an input, verify data ingestion and quality, test the  algorithms, verify communication through the API layer, and reconcile the final output on the data visualization platform with expected  output • Check for system security, i.e., static and dynamic security testing • Conduct user interface and regression testing of the systems

---

  Page: 5 / 8

---

 External Document © 2018 Infosys Limited Use case 3: Testing ML-based analytical models Organizations build analytical models for three main purposes as shown in Fig 2 Fig 2: Types and purposes of analytical models  The validation strategy used while testing the analytical model involves the following three steps: •  Split the historical data into ‘test’ and ‘train’ datasets  •  Train and test the model based on generated datasets •  Report the accuracy of model for the various generated scenarios Fig 2: Types and purposes of analytical models  External Document © 2018 Infosys Limited

---

  Page: 6 / 8

---

 External Document © 2018 Infosys Limited Fig 3: Testing analytical models  Organizations build analytical models for three main purposes as shown in Fig 2 Fig 2: Types and purposes of analytical models  Use case 4: Testing of AI-powered solutions Chatbot testing framework • Test the chatbot framework using semantically equivalent sentences and create an automated library  for this purpose • Maintain configurations of basic and advanced semantically equivalent sentences with formal and  informal tones and complex words • Automate end-to-end scenario (requesting chatbot, getting a response and validating the response  action with accepted output) • Generate automated scripts in Python for execution  RPA testing framework • Use open source automation or functional testing tools (Selenium, Sikuli, Robot Class, AutoIT) for mul- tiple applications • Use flexible test scripts with the ability to switch between machine language programming (where  required as an input to the robot) and high-level language for functional automation • Use a combination of pattern, text, voice, image, and optical character recognition testing techniques  with functional automation for true end-to-end testing of applications While testing a model, it is critical to do the following to ensure success:  • Devise the right strategy to split and subset historical dataset using deep knowledge of development model and code to understand how it  works on data • Model the end-to-end evaluation strategy to train and recreate model in test environments with associated components • Customize test automation to optimize testing throughput and predictability by leveraging customized solutions to split the dataset, evaluate  the model and enable reporting

---

  Page: 7 / 8

---

 External Document © 2018 Infosys Limited Conclusion AI frameworks typically follow 5 stages –  learning from various data sources, input  data conditioning, machine learning and  analytics, visualization, and feedback. Each  stage has specific failure points that can  be identified using several techniques.  Thus, when testing the AI systems, QA  departments must clearly define the  test strategy by considering the various  challenges and failure points across all  stages. Some of the important testing  use cases to be considered are testing  standalone cognitive features, AI platforms,  ML-based analytical models, and AI- powered solutions. Such a comprehensive  testing strategy will help organizations  streamline their AI frameworks and  minimize failures, thereby improving output  quality and accuracy.  External Document © 2018 Infosys Limited

---

  Page: 8 / 8

---

 © 2018 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys  acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this  documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the  prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.  For more information, contact askus@infosys.com Infosys.com | NYSE: INFY Stay Connected

***


# Infosys Whitepaper
Title: RPA validation pitfalls and how to avoid them
Author: Infosys Limited
Format: PDF 1.6

---

  Page: 1 / 8

---

 PERSPECTIVE RPA VALIDATION PITFALLS AND HOW  TO AVOID THEM MANOJ AGGARWAL, Delivery Manager, Infosys

---

  Page: 2 / 8

---

 External Document © 2018 Infosys Limited Abstract While many organizations are adopting robotic process  automation (RPA) to increase operational efficiency,  the approach for testing and validating robots is still a  software-first strategy. The lack of the right testing and  validation strategy can result in under-performing robots  that are unable to meet the desired business and efficiency  outcomes. This paper outlines erroneous RPA testing  strategies and provides an effective approach to RPA  validation.<|endoftext|>External Document © 2018 Infosys Limited

---

  Page: 3 / 8

---

 External Document © 2018 Infosys Limited Introduction To execute any business operation or workflow in large enterprises, services personnel or business teams usually work with multiple IT  applications. Robotic process automation (RPA) can automate these business workflows across multiple and disparate applications using  software robots that mimic the actions of human users as depicted below: The increased adoption and criticality of RPA for business process automation makes it essential to design and build RPA robots that exceed the  level of productivity and quality delivered by human users. However, many RPA robots tend to underperform due to poor design and improper  implementation in addition to workload scheduling decisions. Further, QA teams struggle to identify these issues during robot validation  primarily due to recurring test strategy and test execution challenges as described below: What to test How to test Where to test Focus on validating IT application functionality: Since business process  automation is always implemented on ap- plications used by the business, it is fair to  assume that these applications are tested  and work properly in production. However,  QA teams continue to invest significant  effort to validate application behavior  like querying the database to verify data  updates, checking error messages in case of  negative scenarios, etc.  Missing non-functional requirements  driven by the operating environment:  Sometimes, validation teams miss out on  capturing and validating operating envi- ronment requirements like business SLAs,  process execution window and dependent  activities.<|endoftext|>Validation of robots as software: One of  the most striking challenge is that validation  teams treat robotic process testing as software  validation. Their approach focuses on test  steps automation, validation of the Robot in- puts with database and re-verification of steps  performed by Robots. While some of these  validation steps are relevant, teams tend to  miss the key validations required for seamless  and predictable functioning of RPA robots.<|endoftext|>Automating RPA testing:  There are many  instances where efforts are made to auto- mate RPA testing using test automation tools  without much clarity on what needs to be  automated when RPA itself works in an auto- mated manner.<|endoftext|>Validation in incorrect environments:  Many QA teams begin validating RPA  robots in SIT and other lower environments  where the application baseline is usually  not in sync with the production environ- ment. This asynchrony results in robots fail- ing to perform in the production environ- ment during the first few runs.<|endoftext|>

---

  Page: 4 / 8

---

 External Document © 2018 Infosys Limited A successful example of robot validation  A good example for RPA validation can be gleaned from the manufacturing industry that has successfully deployed physical robots over many  years with a very high level of quality and predictability. On comparing how physical robots are tested with how RPA robots should be tested,  the following key testing attributes emerge:  1. The robot’s ability to perform as per the instructions   2. The robot’s ability to perform tasks autonomously 3. The robot’s ability to perform at higher efficiency 4. The robot’s ability to handle exceptions gracefully A recommended approach for RPA validation   Combining the knowledge of software validation with the processes followed by the manufacturing industry for robot validation provides a  fitting approach for RPA validation that overcomes the testing challenges mentioned above: What to test How to test Where to test Validation scenarios: Since robots are  designed to mimic business workflows, com- panies can re-use/create test cases similar to  UAT test cases for RPA validation. The focus  should be on validating business flows and  business exceptions rather than application  functionality Non-functional requirements validation in  the operating environment: Teams should  capture non-functional requirements like  the ability to operate autonomously, SLA  delivery efficiency and expected volume to  be processed within preset timeframes Functional validation of robots: Validation  teams should set up the input case data for  all business cases within the determined  scope and then begin robot processing.  They should also validate the outcomes  and re-runs in cases of errors/unexpected  outcomes Exception scenarios: Teams should set  up the inputs case data with exception  scenarios and validate the robot’s ability  to identify and report these exceptions for  human intervention. They should also re-run  and validate these tests in case of errors/un- expected outcomes UAT or higher environments: QA teams  should perform validation of RPA only in UAT  or higher environments to prevent issues  arising from incorrect application versions or  environment set-up Stabilization in production: Teams should  train robots with simple cases and at low  volume during initial runs and gradually  increase complexity and volume to ensure  minimal impact in case of any processing  mismatch

---

  Page: 5 / 8

---

 External Document © 2018 Infosys Limited External Document © 2018 Infosys Limited

---

  Page: 6 / 8

---

 External Document © 2018 Infosys Limited Skill recommendations  RPA validation teams should have a deep  understanding of business processes  across regular and exception workflows to  ensure that robots are tested for all possible  business scenarios. Additionally, the teams  should be well aware about the scheduling  mechanism and controller team operating  model of the RPA robots so that they can  validate the operational efficiencies of RPA  robots before deployment into production.  Each RPA validation team should also  leverage a shared team of experts who  have in-depth knowledge of object  design and error handling mechanisms.  This is important to maintain focus on the  performance tuning of robots to meet the  operational efficiency requirements.<|endoftext|>As the RPA ecosystem continues to  evolve, one can expect more complex  robot implementations with optical  character recognition (OCR) and natural  language processing (NLP) inputs as well  as AI integrations. Moreover, robots will  increasingly compete for shared resources  like server time and application access to  complete the business processes. In such  a scenario, RPA validation teams should  continue acquiring in-depth knowledge  on evolving validation needs and improve  the strategies to support these complex  business requirements.  Exit criteria  Before a robot is deployed into production,  it is important to check that it has  been validated to exceed the business  benchmarks and is predictable. Here  are some key parameters that should  be considered as the exit criteria for  robots before they are deployed into live  production: • The adherence to business SLAs should  be higher than that of manual processing  SLAs • The number of business processing  exceptions should be equal to or less  than the number of manual processing  exceptions • The total number of cases processed  without human intervention should be  equal to or greater than 95% of the in- scope cases  • Robot availability should exceed 98%  Typically, robots meeting these criteria are  well-placed to take over business processing  responsibilities from human users and  are proven to deliver the desired business  benefits.<|endoftext|>

---

  Page: 7 / 8

---

 External Document © 2018 Infosys Limited Conclusion RPA has the potential to transform how  organizations execute business workflows  by enabling higher efficiency, faster  outcomes and almost error-free operations.  One of the key success drivers is the  right testing strategy for RPA validation.  Validating robots using existing software  testing models is ineffective because it  often results in non-performing or under- performing assets. To overcome this  challenge, organizations need a strategy  that tests the robot’s capacity to work  autonomously, handle exceptions well,  operate at higher efficiency, and follow  preset instructions. With the right validation  approach, skills and exit criteria, RPA can  help organizations meet the desired  business outcomes of error-free, predictable  and efficient business operations.<|endoftext|>External Document © 2018 Infosys Limited

---

  Page: 8 / 8

---

 © 2018 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys  acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this


Continue # Infosys Whitepaper
 document. Except as expressly permitted, neither this  documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the  prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.  For more information, contact askus@infosys.com Infosys.com | NYSE: INFY Stay Connected

***


# Infosys Whitepaper
Title: Scaling continuous testing across the enterprise
Author: Infosys Limited
Format: PDF 1.6

---

  Page: 1 / 8

---

 WHITE PAPER SCALING CONTINUOUS TESTING  ACROSS THE ENTERPRISE Abstract Over the years, organizations have invested significantly in optimizing  their testing processes to ensure continuous release of high-quality  software. Today, this has become even more important owing to digital  transformation. This paper examines some of the critical features of setting  up a robust continuous testing practice across the enterprise. It considers  the technology and process standpoints and provides guidelines to ensure  a successful implementation from inception to delivery of high-quality  software.<|endoftext|>

---

  Page: 2 / 8

---

 External Document © 2020 Infosys Limited Establishing a continuous testing (CT)  practice is an organization-level change  that cannot be executed with a ‘big bang’  approach. It should be conceptualized,  implemented and allowed to mature at a  program level. Once continuous testing  gains inroads into various programs across  the enterprise, it can then become the  new norm for enterprise software delivery.  To enable a seamless journey towards  organization-wide continuous testing, the  gradual change must be sponsored and  supported by senior IT leadership.  To begin with, enterprises must set up a  program-level center of excellence (CoE)  and staff it with people who have a deep  understanding of automation. This team  should be responsible for: •   Identifying the right set of tools for CT  •   Building the right solutions and  practices that can be adopted by the  program •   Integrating automated tests with  the DevOps pipeline for continuous  testing These action points lay the foundation for  establishing CT at a program level. They  can subsequently be improved and aligned  depending on the enterprise’s needs.  Introduction External Document © 2020 Infosys Limited

---

  Page: 3 / 8

---

 External Document © 2020 Infosys Limited The journey Once the CoE has been set up, enterprises must focus on expanding the practice of continuous testing within all program-level agile  teams across the organization. This can be done by promoting the existing program-level CoE into an enterprise-level CoE with broader  responsibilities.  The primary goal of the enterprise-level CoE should be: •   To ensure CT adoption by all scrum teams •   To establish real-time reporting, metrics and measurement for faster adoption •   To identify areas with zero/low automation and provide support Enterprises can also accelerate CT adoption by extensively showcasing the benefits realized from programs.<|endoftext|>Fig 1: Continuous testing journey of an enterprise External Document © 2020 Infosys Limited Transform  from   Program COE – Org COE Branding at Org  Level –solutions,  metrics Matured CT at  Program level Help Program  Scrums to adopt  CT Establish Program  Level CT COE Challenges during enterprise adoption   Extending continuous testing operations across the enterprise can be daunting. Enterprises should be prepared to face resistance to change  arising from higher operational costs and limited technical knowledge. Some of the key challenges of CT adoption and the ways to address  them are listed below: Challenges during enterprise adoption The way forward Multiple automation solutions – Teams often devise their own ways  to implement frameworks, leading to redundant code, confusion  and wastage During the initial assessment phase, enterprises can invest in  developing a solution that is capable of porting with existing  enterprise code Low technical knowledge for automation – QA organizations often  don’t have the necessary skills to build and maintain automation  scripts Focus on talent enablement through brown-bag meetings, training  workshops and support. Enterprises can also use script-less and  model-based automation tools Pushback from certain lines of business (LOBs) – Most enterprises  use disparate technology stacks and execution methodologies. It is  more challenging to adopt CT in some domains on legacy such as  mainframe systems, PowerBuilder environments and batch jobs Build a robust automation framework that supports different  technology stacks for an improved usability experience. Define  different goals for different technology stacks (like lower automation  goals for legacy and higher automation targets for APIs) to ensure  parity across teams during adoption Limited funding – Many customers view continuous testing as an  expense rather than an investment since it includes automation  development as well as integration costs Enable individual QA teams to adopt in-sprint automation, thereby  transferring the high initial cost to individual projects. This reduces  the overall enterprise CT cost to the expense of integrating  automation with the DevOps pipeline

---

  Page: 4 / 8

---

 External Document © 2020 Infosys Limited Federated Operating Model Tool Standardization Enterprise Solutions & Process  Automation Enterprise  Continuous  Testing CT Practices & Processes & Enterprise adoption Assess and Identify the  Best Tool sets Build The right Solution  for Automation Training, Docs & Videos,  Org Level CT metrics Centralized COE, with  Distributed Automation  teams Infosys approach on establishing CT 1. Establish a federated operating model    Based on our past experience across several engagements, Infosys has identified four main dimensions that are essential to establishing and  operating a successful continuous testing practice across any enterprise. These dimensions are described below.<|endoftext|>A robust operating model is vital for  any practice. Earlier models comprising  functional testing teams and shared  automation teams failed because these  hamper functional teams from achieving  in-sprint/progressive automation within  2-3-week sprint windows. Thus, the way  forward is to embrace automation as a  culture and consider it as a business-as- usual process for anything that a tester  does.<|endoftext|>Care must be taken to ensure that generic  and complex solutions are built and  maintained by a centralized CT CoE team.  In their daily tasks, regular QA teams must  adapt to artifacts developed by the CoE  team. Through this federated method of  operations, enterprises can simply deploy a  few people with specialized skills to create  CT artifacts, thereby supporting regular  testers with relative ease.<|endoftext|>Fig 2: Key dimensions for establishing a successful CT practice Fig 3: Structure, roles and interactions in a federated operating model with centralized CoE • Liaise with CI-CD to integrate CT with pipeline • Develop generic automation artifacts – Frameworks / processes /  solutions • Provide support, handholding • Metrics & reporting • Build automation tests for supported apps • Leverage artifacts created by CT COE • Request for enhancements and provide suggestions • Request for help and support for CT adoption Enterprise QA teams Org Level CT COE Team External Document © 2020 Infosys Limited

---

  Page: 5 / 8

---

 External Document © 2020 Infosys Limited 2. Standardize tools  3. Build enterprise and process automation solutions The process of standardizing tools should  be objective. It should first assess the  existing enterprise tools, identify those  that can be reused and procure new  tools for CT. Here are some important  considerations when evaluating any tool: •   Tool fitment – Ensure the tools are the  right ones for the task they have to  service •   Tool license and operational cost – This  should fall within the client’s budget •   Tool scalability – It should be  capable of scaling to meet future  organizational needs •   Tool integration – In a DevOps world,  tools should be easily compatible with  other tools and pipelines •   Tool support – There should be  adequate online and professional  support services for the tool Currently, there are many tools available in  the market that support the various stages  of the software development lifecycle  (SDLC). Enterprises must be careful to  onboard the right set of tools that will aid  software development and QA.<|endoftext|>The automation solutions built to aid  and enable CT are at the heart of any  robust continuous testing practice. These  solutions should encompass every phase  of the software testing lifecycle. Some  critical automation artifacts that must be  developed to sustain a robust CT practice  are: a) Automation frameworks – This is the  basic building block that aids automation  and enables CT across an enterprise, as  shown below in Fig 4.<|endoftext|>b) Distributed and parallel test execution  approaches – Speedy test execution is  critical to accelerate software delivery. As  the volume of automated tests increases,  enterprises should adopt distributed and  parallel execution either by onboarding  off-the-shelf tools or by building custom  solutions as per enterprise requirements  c) Test data automation – Testers spend a  significant amount of time setting up test  data. Solutions should be built to automate  the manufacturing, cloning/masking,  mining, and management of test data from  requisition to provisioning d) Process and environment automation  – This involves automating all test and  defect management-related processes.  Environment provisioning automation  is essential to make the entire CT  practice cost effective and manageable  at an enterprise scale and some viable  options are cloud-based or infrastructure  virtualization solutions.<|endoftext|>It is important to note that automation  solutions are simply a critical


Continue # Infosys Whitepaper
 subset of all  the solutions that should be developed  as a part of establishing a successful CT  practice. Care must be taken to prioritize  developing these solutions and weighing  their benefits according to the needs of the  enterprise.<|endoftext|>Fig 4: Characteristics of a robust automation framework Controllable – Centrally  managed Scalable – Ready for Future Agnostic – Agility for Change Multi Tech. Support – Enterprise coverage Reusable – Write once run  many times Portable – Run anywhere Open Integration – With other tools Ease of Use – Script less to  the extent possible

---

  Page: 6 / 8

---

 External Document © 2020 Infosys Limited 4. Establish enterprise CT processes and drive adoption Successful products are the sum total of  their technologies, processes and best  practices, which make products attractive  and easy to adopt. Having explored the  technical solutions and artifacts for CT,  let us examine some of the critical CT  processes and strategies for enterprise  adoption.<|endoftext|>a) CT processes – Robust processes act  as a pointer for all who embark on the  CT journey. The key processes to be  established are: • Day-wise activity tasks for helping teams  adopt in-sprint automation, as shown in  Fig 5  • CT metrics, measurement and reporting  like overall automation coverage, in-sprint  automation percentage, percentage of  defects found from automation, execution  time per run, code quality, and code  coverage  b) Adoption strategies – Implementing  CT across the enterprise can be facilitated  through the following ways: •   Providing organization-level branding  like communications, mailers and  workshops as well as talent enablement  and support through brown-bag meetings,  demos, trainings, self-learning videos,  playbooks, etc.  •   Accelerating adoption through  centralized CT metrics reporting that  introduces healthy competition, faster  adoption and early identification of  problem areas We, at Infosys, believe these dimensions  can help enterprises set up and operate  a mature continuous testing practice.  Nevertheless, it is important to customize  these according to the needs of each  organization.  • Baselined requirements • Script development 1-3 • Get artifacts from dev such as  html fles, wireframes for  scripting • Daily code deployment  for incremental changes • Unit testing of scripts • QA code deployment for  changes • Automated script execution  • Address code fxes for  defects • Rerun tests and signof 4 5-7 8 9-10 Fig 5: Daily tasks during a 10-day sprint External Document © 2020 Infosys Limited

---

  Page: 7 / 8

---

 External Document © 2020 Infosys Limited Conclusion Establishing a continuous testing practice  across the enterprise comes with its own  challenges and complexities. These include  insufficient knowledge of automation,  limited funding, resistance to change, and  disparate technologies. When developing  a strategy for enterprise-wide adoption of  CT, Infosys recommends paying attention  to four critical dimensions. These include  creating a federated operating model,  standardizing tools, building automation  solutions, and establishing CT processes.  Through our approach, enterprises can  benefit from a roadmap to successfully  implement CT with a dedicated center  of excellence. The successful adoption of  continuous testing mandates changing  how people work and embracing  automation as part of the organizational  culture and business as usual.<|endoftext|>External Document © 2020 Infosys Limited

---

  Page: 8 / 8

---

 © 2020 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys  acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this  documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the  prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.  For more information, contact askus@infosys.com Infosys.com | NYSE: INFY Stay Connected Mohanish Mishra, Senior Project Manager – Infosys Limited About the
Author

***


# Infosys Whitepaper
Title: IoT Security Assurance
Author: Infosys Limited
Format: PDF 1.7

---

  Page: 1 / 8

---

 VIEW POINT Abstract IoT is one of the rapidly growing emerging technologies. It is  entering in all walks of our life with connected cars, connected smart  homes, connected healthcare, wearables, etc. and exchanging large  amount of sensitive/PII information over internet. This increases the  security risk of IoT devices. In this document we are proposing an  approach for security assessment of various IoT components.<|endoftext|>IOT SECURITY ASSURANCE

---

  Page: 2 / 8

---

 Introduction New testing processes and tools are  developed continuously to improve the  quality of software. Today, the IT industry  is gaining momentum in agile deliveries  and development and operations  (DevOps), which is creating new pos- sibilities by integrating development,  test, and operations teams. To keep pace  with these changes, testing processes  and tools need transformation such that  testing platforms can be accessible to all  stakeholders and made simple.    External Document © 2018 Infosys Limited

---

  Page: 3 / 8

---

 User  Interface Application Data Collector Device IoT Applications IoT Systems Architecture It’s evident from the above figure that IoT devices have multi-dimensional usage and are omnipresent. Well, this wide variety of IoT applica- tions poses some interesting use cases with respect to security of data. Suppose you’re using a mobile app for unlocking and controlling  peripherals like air conditioning, music system etc., of a vehicle. If someone is able to intercept into the communication channel, say Wi-Fi,  used between app and peripheral receiver. The attacker would be able to pose threat to the owner as well as the vehicle.<|endoftext|>I. Device - Sensors, actuators for monitoring  and notifying the events.<|endoftext|>II. Collector – systems to collect and  pre-process the data sent by sensors III. Data – repository of data accumulated  after pre-processing, it can be local or  cloud as well IV. Application – processes the data  according to the required usage V. User Interface – Web or mobile app  interface which provides relevant  information to the user VI. Communication Channel – wired or  wireless communication link between two  layers, it may be entirely absent for a locally  connected layer, say when collector and  data layers reside on same system.<|endoftext|>Figure 2: IoT Architecture Communication Channel IoT Applications Ranking Figure 1. Ranking of the IoT applications External Document © 2018 Infosys Limited

---

  Page: 4 / 8

---

 Components Attack Surface Devices (Sensors,  Gateways) Device memory, ﬁrmware, physical interfaces like USB ports, web  interfaces, admin interfaces, Update Mechanism Communication  Channel Device Network traﬃc  using LAN, Wireless (Wi-Fi, ZigBee, Bluetooth) Cloud Interface Getting access to sensitive data/PII stored on cloud by Injection attacks,  weak passwords or default credentials, Insecure Transport encryption.<|endoftext|>Application Interface  (Web and mobile) Getting access to sensitive data or PII by exploiting vulnerabilities like  OWASP web and mobile Top 10, in application interfaces.<|endoftext|>Attack Surfaces  The Internet of Things infrastructure can be  divided mainly into four components, 1. Devices (Gateways, Sensors, Actuators) 2. Communication Channel (Wi-Fi,  Bluetooth) 3. Cloud Interface 4. Application Interface (mobile and/or  web)  Latest attacks  The insecure implementation of IoT devices  are routinely being hacked and even used  as accessories in cyber-attacks.<|endoftext|>1. Hotel Room Locks prone to Hacking  Device: -  Onity United Technologies are leading  supplier of electronic locking system. A  Mozilla developer Cody found vul- nerabilities in the locks in 2012. The  device created by Cody reads the lock’s  memory and gets the cryptographic key  information. It sends that information to  the door lock which allows the hacker to  gain access to the room.   2. Massive DDoS attacks zombies 25,513  CCTV cameras: -   Researchers from Sucuri have claimed  that CCTV cameras can and are being  used for DoS attacks. The attackers use  these cameras as botnets. The attack  may last for days and could surge to a  several thousand HTTP requests per  second. Following pie chart depicts the  distribution of CCTV Botnets.<|endoftext|>Table – IoT components are their attack surfaces  Figure 3. CCTV DDoS Botnet Geographic Distribution (Data Courtesy: https://blog.sucuri.net/2016/06/large-cctv-botnet- leveraged-ddos-attacks.html)  25% 24% 12% 9% 8% 6% 5% 5% 2% 2% 2% Rest of the World Taiwan USA Indonesia Mexico Malasia Israel Italy Vietnam France Spain External Document © 2018 Infosys Limited

---

  Page: 5 / 8

---

 Infosys security assessment  approach for IoT Following practices should be followed while  designing an IoT system, to ensure security: - If any other protocols like ZigBee is  used, then following methods can be  implemented to mitigate any possible  security issues:  1. Implement AES Encryption. It provides  confidentiality as well as integrity.<|endoftext|>2. Implement Master Keys to secure Key  Establishment Procedure.<|endoftext|>3. Implement Link Keys to encrypt the  information sent across nodes.<|endoftext|>4. Implement Network Keys to authen- ticate and validate each device which  attempts to join the network.<|endoftext|>If MQTT protocol is being used, then follow  following methods.<|endoftext|>1. A firewall with sophisticated ruleset  should be implemented for every  connection to a MQTT broker.<|endoftext|>2. Block all the UDP packets as MQTT  uses TCP.<|endoftext|>3. All the ICMP packets should not be  blocked as response to PING and  TRACEROUTE will be hampered.  Instead, investigation of ICMP packets  is good approach.<|endoftext|>4. Traffic to any ports which are not  needed for the MQTT system should be  blocked. Following are MQTT ports • 1883: This is the default port for MQTT  over TCP.  • 8883: This is the default port for MQTT  over TLS.  • Use MQTT over TLS for all communica- tions.  • Use updated software for all the  component as they will be fixed for  older security vulnerabilities.] • Apart from above guidelines imple- mentation of Demilitarized Zones and  Load Balancers will further strengthen  the security of the system.  3. Nissan Leaf electric cars hack vulnerabil- ity disclosed: -  2016 has proved out to be a bad year  for IoT security after all. In early 2016  Troy Hunt, and Australian web security  expert demonstrated how Nissan Leaf’s  companion app can be used to hack  the vehicle. Even the app was optional,  as a simple web request via a browser  was able to control vehicle AC, Heating  system and may also reveal the owner’s  identity, if its VIN was known. This was  proved by the researcher while sitting  on a computer in Australia, controlling  systems of a vehicle that belonged to an  acquaintance of his in United Kingdom!  Though, the criticality of the issue may  not be life threatening but could still be  used to drain car battery for least to say,  which in turn can leave the owner in  perilous situations.<|endoftext|> Thus, the security incidents discussed  are examples of how lack of security  measures in IoT devices can not only  compromise personal security but also  can be leveraged to impact security of  other internet services as well.<|endoftext|>Figure 4. Extent of attack on Nissan Leaf External Document © 2018 Infosys Limited

---

  Page: 6 / 8

---

 Recommended security testing approach for each layer of IOT stack is as follows Sensor : Hardware Security threats Security assessment test case L2: Insuﬃcient Authentication/
Authorization L3: Insecure Network Services L5: Privacy Concerns L8: Insuﬃcient Security Conﬁgurability L10:Poor Physical Security Check for Password Complexity and Password Recovery mechanism for device Check for poorly Protected Credentials and ineﬃcient two Factor Authentication Check for Role Based Access Control Check for open ports, check if there are any unnecessary ports utilized.<|endoftext|>Check if getting access to device memory to get sensitive/personal data stored,  encryption keys, certiﬁcates is allowed Check if Firmware extraction and modiﬁcation is possible Check for User or Admin Command Line Interface issues Privilege escalation Check if device can be reset to insecure state or default state Check if device or internal memory can be accessed via USB ports and SD cards Sensor : Software Security threats Security assessment test case L3: Insecure Network Services L9:Insecure Firmware/ Software


Continue # Infosys Whitepaper
 Radio communication analysis between device and the gateway by attacking ZigBee,  zWave, 6LoWPAN Attacking Bluetooth Low Energy (BLE) Checking if device has direct connection - connecting to mobile app which is in same  network  Check for ﬁrmware Update Functionality Check if ﬁrmware Contains Sensitive Information Check if ﬁrmware update functionality is using encryption and secure communication  and update ﬁle encrypted Check for insecure or misconﬁgured services like FTP, Telnet, TFTP, Finger, SMB, e.g.  misconﬁgured NAT-PMP services, hard-coded Telnet logins Gateway (Raspberry Pi: USB) Security threats Security assessment test case L2: Insuﬃcient Authentication/
Authorization L3: Insecure Network Services L5: Privacy Concerns L8: Insuﬃcient Security Conﬁgurability L10:Poor Physical Security Check for Password Complexity and Password Recovery mechanism for device Check for poorly Protected Credentials and ineﬃcient two Factor Authentication Check for Role Based Access Control Check for open ports, check if there are any unnecessary ports utilized.<|endoftext|>Check if getting access to device memory to get sensitive/personal data stored, encryption  keys, certiﬁcates is allowed Check if Firmware extraction and modiﬁcation is possible Check for User or Admin Command Line Interface issues Privilege escalation Check if device can be reset to insecure state or default state Check if device or internal memory can be accessed via USB ports and SD cards External Document © 2018 Infosys Limited

---

  Page: 7 / 8

---

 Message broker Security threats Security assessment test case L4: Lack Of Transport Encryption Insecure Data Storage Check for sensitive data stored on Kafka as it does not support encryption of data at rest Check if conﬁguration ﬁles can be accessed and modiﬁed Cloud Interface  Security threats Security assessment test case I6: Insecure Cloud Interface OWASP Cloud Top 10 vulnerabilities Arbitrary Code Execution Check for Insuﬃcient authentication, lack of transport encryption and account enumeration  to access data or controls via the cloud website.<|endoftext|>Check if ﬁrewall is conﬁgured (if the master of the cluster is exposed to the internet without having any ﬁrewall in between  then anyone with access to the master URI can submit jobs to the cluster remotely) Check for arbitrary code execution Check for OWASP Cloud Top 10 risks R1: Accountability & Data Risk R2: User Identity Federation R3: Regulatory Compliance R4: Business Continuity & Resiliency R5: User Privacy & Secondary Usage of Data R6: Service & Data Integration R7: Multi-tenancy & Physical Security R8: Incidence Analysis & Forensics R9: Infrastructure Security R10: Non-production Environment Exposure Application-Web Interface Security threats Security assessment test case I1: Insecure Web Interface OWASP Web Top 10 vulnerabilities Check for OWASP web Top 10 issues A1:Injection A2:Broken Authentication and Session Management A3:Cross-Site Scripting (XSS) A4:Insecure Direct Object References A5:Security Misconﬁguration A6:Sensitive Data Exposure A7:Missing function level control A8:CSRF A9:Using vulnerabilities from known unknown components A10Unvalidated Redirects and Forwards External Document © 2018 Infosys Limited

---

  Page: 8 / 8

---

 Conclusion References: IoT is no doubt a fascinating, yet emerging technology. The prioritization of rapid  development over security by the developers has caused rise of new IoT vulnerabilities.  A large number of IoT devices have already become victims of hacks, botnets and other  attacks constantly. Proper security frameworks for IoT, like.NET, Java, Android and iOS,  should be made available. IoT solution developers must have security know how. IoT  application development should follow secure development life cycle, security tests  have to be mandatory. In addition to these, advance approaches like machine learning  can also be applied to ensure the IoT security.<|endoftext|>1. http://www.forbes.com/sites/thomasbrewster/2014/11/07/car-safety-tool-could-have-given-hackers-control-of-your-vehicle/#2b98d1ef21b0 2. http://www.computerworld.com/article/2487425/cybercrime-hacking/target-breach-happened-because-of-a-basic-network-segmentation-error.html 3. http://www.ey.com/Publication/vwLUAssets/EY-cybersecurity-and-the-internet-of-things/$FILE/EY-cybersecurity-and-the-internet-of-things.pdf  4. http://h30499.www3.hp.com/t5/Fortify-Application-Security/HP-Study-Reveals-70-Percent-of-Internet-of-Things-Devices/ba-p/6556284#.VHMpw4uUfVc  5. http://www.ey.com/Publication/vwLUAssets/EY-cybersecurity-and-the-internet-of-things/$FILE/EY-cybersecurity-and-the-internet-of-things.pdf  6. http://www2.deloitte.com/content/dam/Deloitte/global/Documents/Technology-Media-Telecommunications/gx-tmt-Iotecosystem.pdf  7. http://internetofthingsagenda.techtarget.com/info/getstarted/Internet-of-Things-IoT-Security-Threats  8. http://internetofthingswiki.com/iot-trends-in-2016/300/  9. http://internetofthingswiki.com/iot-trends-in-2016/300/   10.https://iotsecuritywiki.com/ About the authors •   Amitesh Gaurav is the Systems Engineer working with Infosys Center for Emerging Technology Solutions group. He works as Security  analyst. His focus areas are Web, Mobile and IOT Application security assurance.<|endoftext|>•   Jayaprakash Govindaraj is the Senior Technology Architect, and leads Security CoE at Infosys Center for Emerging Technology Solutions  group. His focus areas are Web, Mobile and IOT Applications Security Assurance, Secure Development and Managed Security Services.<|endoftext|>Application-Web Interface Security threats Security assessment test case I7: Insecure Mobile Interface OWASP Mobile Top 10 vulnerabilities Check for OWASP Mobile Top 10 issues M1: Weak Server Side Controls   M2: Insecure Data Storage   M3: Insuﬃcient Transport Layer Protection   M4: Unintended Data Leakage   M5: Poor
Authorization and Authentication   M6: Broken Cryptography   M7: Client Side Injection   M8: Security Decisions Via Untrusted Inputs   M9: Improper Session Handling   M10: Lack of Binary Protections  © 2018 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys  acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this  documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the  prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.  For more information, contact askus@infosys.com Infosys.com | NYSE: INFY Stay Connected

***


# Infosys Whitepaper
Title: Deployment of Public-Key Infrastructure in Wireless Data Networks
Author: Infosys Technologies
Format: PDF 1.4

---

  Page: 1 / 10

---

   P A Parametric Approach for Security Testing of Internet Applications   Arun K. Singh   E-Commerce Research Lab.    Anand J. Iyer,   Infosys Software Labs. for Test Execution and Research.    Venkat Seshadri  Product Competency Center    Infosys Technologies Limited  Electronic City, Bangalore, India  {arunks, Anandj_iyer, venkatar}@infy.com  Abstract. Security is one of the prime concerns for all the Internet applications. Often it  is noticed that during testing of the application, security doesn’t get due focus and what- ever security testing is done, it is mainly limited to security functionality testing as cap- tured in the requirements document. Any mistake at requirement gathering stage can  leave the application vulnerable to potential attacks. This paper discusses the issues and  challenges in security requirement gathering and explains how it is different from normal  functionality requirement gathering. It presents an approach to handle different factors  that affect application security testing.   1   Introduction  As compared to traditional client-server or mainframe-based applications, Internet applications have  different security requirements [1]. For any Internet application, there is a need to provide an unre- stricted global access to the user community.  This also brings in the threat of outside attack on the  system. The hackers may try to gain unauthorized access to these systems for a variety of reasons and  disturb the normal working of the application.     © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 2 / 10

---

   Presented at the 3rd International Software Testing Conference, India  The security of Internet applications necessarily means prevention of two things: destruction of in- formation and unauthorized availability of information. The broad spectrum of application security  can be expressed as “PA3IN” viz. Privacy, Authentication, (
Authorization, Audit Control,) Integrity  and Non-repudiation.     The first issue in application security is to provide data privacy to the users. This means protecting  information confidentiality from the prying eyes of unauthorized internal users and external hackers.  Before allowing the access permissions to the users, it is essential to ensure the legitimacy. The proc- ess of identifying users is called authentication. Establishing the users’ identity is only half the battle.  The other half is access control/ authorization which means attaching information to various data  objects denoting who can and who cannot access the object and in what manner (read, write, delete,  change access control permissions, and so forth). The third ‘A’ is for audit control which means  maintaining a tamper proof record of all security related events. And then when the data is in transit  across the network, we need to protect it against malicious modification and maintain its integrity.  The non-repudiation means the user should be unable to deny the ownership of the transactions made  by them. It is implemented using Digital Signature [3].   This paper presents an approach for testing security of Internet applications.   2 Issues in Testing Security of Internet Applications  The two main objectives of an application security testing are to:  1.  Verify and validate that the security requirements for the application are met.  2.  Identify the security vulnerabilities of the application under the given environment   For security testing of an application, the first step is to capture requirements related to each of the  security issue. This could not only be a tricky and painstaking exercise because the security require- ments are seldom known very clearly at the time of project initiation. The traditional Use case ap- proaches have proven quite useful in general requirements engineering, both for eliciting require- ments and getting a better overview of the stated requirements [4]. However security requirements  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 3 / 10

---

   Presented at the  essentially need to concentrate on what should not happen in the system and this cannot be captured  by traditional Use case approach.     Also, the security model of Internet applications has undergone many changes. The traditional model  for securing an application from outside elements mainly relied on access control. This model was  based on creating a hard perimeter wall around the system and providing a single access gate that can  be opened only for authenticated users. This security model has worked well with most of the simple  Internet based applications. The gateway here refers to a firewall that classifies all users as “trusted”  or “untrusted”. In this simplistic model of security, every “trusted” user who is allowed to cross the  gate, gains access to every portion of one’s business and no further security checks are done. As the  requirements for security increases, the applications need to implement a fine-grained security. In- stead of allowing the trusted user to access every portion of the business, the modern security models  divide the business domain into many regions and ensure different levels of security for each region.  This means creating a security perimeter for each region. The first level of security check can not be  very rigorous as one would want to let in prospective customers, vendors and service providers as  quickly as possible. Most of the Internet applications today have multiple security regions with dif- ferent levels of security and these regions could be nested or overlapping with other regions within  same single application. While developing an approach for testing security of these kinds of Internet  applications, a proper strategy planning is very much essential. It is better to plan security testing at  requirement gathering stage itself because there are many issues, which cannot be captured later on  by usual methods of testing [2]. One such example is testing for denial of service attack.    The issues discussed above make security testing of Internet a very challenging task. In the next sec- tion, a parametric approach to security testing is presented and each step is explained with the help of  a sample example.  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 4 / 10

---

   Presented at the 3rd International Software Testing Conference, India  The Parametric Approach for Security Testing  As discussed in previous section, many of the security parameters cannot be captured and tested us- ing traditional approach. In the proposed parametric approach for security testing, before we start the  requirement gathering, a template to enlist all security parameters are created. This task can be  achieved in four steps as described below:  1.  Create an exhaustive list of all security issues in the application   2.  Identify all possible sub parameter for each of these issues  3.  List all the testing activities for each sub-parameter  4.  Assign weightages corresponding to the level of security and priority.    Once this template is ready, it can be used to streamline each stage of testing lifecycle. The lifecycle  process of security testing is similar to the software development lifecycle process. In the proposed  approach, the security testing lifecycle stages are as follows:  Test bed  Implementation Test Design and  Analysis  Capture  security  test  requirements Test Reporting  Figure 3.1: Security Testing life cycle    a)  Capture security test requirements  b)  Analyze and design security test scenarios  c)  Test bed implementation  d)  Interpreting test reports   In the following sub-section each of these stages are explained.  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 5 / 10

---

   Presented at the 3rd International Software Testing Conference, India  3.1 Parametric Approach to Capture Security Test Requirements    To define the scope of security testing, check the stated requirement against the parametric template.  This will help in identifying all the missing elements or the gaps in security requirement capture. The  next step would be to allocate appropriate weightages to the various security sub parameters. These  weightages would typically be determined by the nature of business domain that the application ca- ters to and the heuristic data available with the security test experts.     For example, consider an e-shopping application for an online store. The key security requirement for  such an application would be to ensure that the user who provides his own payment information is  indeed, who he claims to be and to make sure that the information provided is correct and is not visi- ble to anyone else. From that perspective, authentication, confidentiality and non-repudiation are the  key parameters for consideration. One possible set of weightages for these parameters could be 0.5,  0.25 and 0.25 respectively (on the scale of 0 to 1). This is in line with the impact; a security breach  could have on the business.    The advantage of assigning appropriate weightages is that it helps to optimize the test scenarios for


Continue # Infosys Whitepaper
  each of these parameters. On the next page, table 3.1 presents a sample parametric table. The snap- shot in table 3.1 only comprises of those parameters that are relevant for testing the application  requirement. The main security parameters, as listed in the table 3.1 are - Authentication,
Authoriza- tion, Access control, Non-Repudiation and Audit control. This list is just a sample list and in actual  scenario there can be many more parameters in the template.     These parameters are further classified according to testing subtask.  For example the requirement of  authentication can be implemented using userid/password, smart card, biometric and digital certifi- cates. And each one of this implementation has different testing requirement. Based on the testing  complexity, appropriate weightages are assigned. The weightages can be determined based on past  experience and the series of interactions conducted with the customer during the requirement gather- © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 6 / 10

---

   Presented at the 3rd International Software Testing Conference, India  ing stage. Going back to the example, since biometric requires substantial investment from the cli- ent’s side the digital certificates score higher than the other modes of implementing authentication.    The last column holds the priority level assigned to each sub-parameter. The business domain and the  experience of the test expert determine the priority level assigned to each sub-parameter. For exam- ple, consider a case of Internet banking application. Here the authentication is slightly of higher prior- ity than that of authorization or access control. Where as in case of a b2b application the authoriza- tion and the access control takes higher priority.    All the parametric values are recorded and this constitutes a requirement document outlining the ar- eas and the level of testing that has to be executed.    No.  Parameters  Dimension            (scale of 1 - 5)  Dimension      (scale 1 - 5)  Dimension      (scale 1 - 5)  Dimension       (scale 1 - 5)  Dimension      (scale 1 - 5)  Priority     (scale of  1 – 100)  1  Authentication  -corroboration  that an entity  is who it claims  to be.  No  authentica- tion  (0)  Password  Based  (3)  Smart card  and PIN  (3)  Biometric  (4)  Digital  certificates.  (5)  10  2
Authorization  No
Authoriza- tion  (0)  Role Based   (3)  User Based   (3)  Using Elec- tronic Signa- tures  (for  non- repudiation)  (3)  Role  and  User Based  (4)  10  3  Access Control  No ACL  (0)  Context  Based  (3)  User Based  (3)  Role Based  (3)  All 3  (4)  7  4  Non  repudia- tion  Nothing  (0)  Digital  signature  (3)  Encryption  and digital  signature  (4)      5  5  Audit control  Nothing  (0)  Use  Fire  wall/ VPN  logs  (1)  Custom  made logs  (3)  Employ  managed  security  services  (4)    5  Table 3.1  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 7 / 10

---

   Presented at the 3rd International Software Testing Conference, India  3.2 Parametric Approach to Security Test Analysis and Design   Once the weightages for each security parameter is finalized, the next task is to create test scenarios  for testing the security requirements. The weightages assigned to each sub-parameter goes on to sug- gest the number of scenarios that need to be tested for a particular requirement and the complexity  involved.     The parametric approach also helps in identification of the extent of data that would be needed for  test execution and also the distribution of data based on various scenarios. Also, this will help in  identifying the kind of tools that will be needed and determine the appropriateness of a tool for a  given type of test.    Now the data that is recorded with the parametric template can be tabulated in the below given form:  No.  Parameter  Weightages   (Scale of 1- 5)  Priority   (Scale: 1 – 100)  1  Authentication  3  10  2
Authorization  2.75  10  3  Access Control  2.6  7  4  Non-Repudiation  2.3  5  5  Audit control  2.0  5  Table 3.2    The table 3.2 above is comprised of the security parameter with its corresponding values of Weight- ages and Priority levels. These weightages are the average of weightages assigned for the testing ac- tivity of each sub-parameter as illustrated in the table 3.1. With the availability of these values for  each sub-parameter the test expert is now able to plan the test scenarios to the desired level of priority  and depth.    3.3 Security Test bed Implementation  In case of application level testing the approach of testing the security vulnerabilities should be more  logical. Here logical security deals with the use of computer processing and/or communication capa- bilities to improperly access information. Second, access control can be divided by type of perpetra- © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 8 / 10

---

   P   tor, such as employee, consultant, cleaning or service personnel, as well as categories of employees.  The type of test to be conducted will vary upon the condition being tested and can include:    Determination that the resources being protected are identified, and access is defined for each re- source. Access can be defined for a program or individual.    Evaluation as to whether the designed security procedures have been properly implemented and func- tion in accordance with the specifications.    Unauthorized access can be attempted in on-line systems to ensure that the system can identify and  prevent access by unauthorized sources.    Most of the security test scenarios are oriented to test the system in the abnormal conditions. This  throws up the challenge of simulating the real life conditions that the test scenarios require. There are  various tools that can be used to test various vulnerabilities that the system may have. The choice of  such a tool is determined by the extent of vulnerability, the tool is able to test, Customization of the  tool should be possible and testing of the system using both the external as well as the internal views.  The local users are the internal view and the external users constitute the external views. There are  various programming tools that can be programmed to simulate the various access control breaches  that can happen.     4 Interpreting Security Test Reports    Once tests have been executed and security gaps have been identified, the gaps need to be analyzed in  order to suggest improvements. The reports need to be validated as many a times as possible. The  reports may be misleading and the actual loopholes might be elsewhere or might not exist at all.  The  security test reports are like X-ray reports, where a security expert is essential to decipher the vulner- abilities reported and bring out the actual problem report. There are various security reporting tools.  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 9 / 10

---

   Presented at the 3rd International Software Testing Conference, India  While selecting a security-reporting tool, the following parameters have to be taken in to considera- tion:  1.  Extent if vulnerability, the tool is able to report.  2.  Customization of the tool should be possible.  3.  Testing of the system using both the external as well as the internal views.    Once the analysis of the report is completed a list of vulnerabilities and the set of security features  that are working is generated. The list of vulnerabilities is classified. The classification level of  vulnerability is determined by the amount risk involved in the security breach and the cost of fixing  the defect. Once the classification is done the vulnerability is fixed and retested.  5   Conclusions  A parametric approach to security testing not only, works very well for security requirement capture,  effort estimate and task planning but also for testing different levels of security requirements by dif- ferent components within same application. This approach for requirements gathering comes out to  be real handy in capturing those security requirements, which cannot


Continue # Infosys Whitepaper
 be captured by traditional  means.  This approach enables one to adopt the parametric matrix at each and every stage of security  testing, and improve the values in the matrix after each execution of a project. Due to this capability  of incorporating experiential data, the parametric approach can provide the most effective mechanism  for security testing of Internet applications.  6   Disclaimer  The authors of this report gratefully acknowledge Infosys for their encouragement in the development  of this research. The information contained in this document represents the views of the author(s) and  the company is not liable to any party for any direct/indirect consequential damages.  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

---

  Page: 10 / 10

---

   Presented at the 3rd International Software Testing Conference, India  References  1.  Singh, A.K.: E-Commerce Security. Proc. of National Seminar on E-Commerce & Web En- abled Applications, Harcourt Butler Technological Institute, Kanpur, India (Mar., 2000).  2.  Singh, A.K., Subraya B.M. : Modelling Security at Requirements stage in E-Commerce Pro- jects, Proc. of Information Security Solutions Europe Conference (ISSE 2000), Barcelona,  Spain (Sept., 2000).  3.  Microsoft Product Security – http://www.microsoft.com/ntserver/info/securitysummary.htm  4.  Eliciting Security Requirements by Misuse cases -- Guttorm Sindre, Norwegian Univ. of Sci.  and Tech and Andreas L. Opdahl, Univ. of Bergen, Norway  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.<|endoftext|>

***


# Infosys Whitepaper
Title: Achieving Order through Chaos Engineering: A Smarter Way to Build System Resilience
Author: Infosys Limited
Format: PDF 1.7

---

 Page: 1 / 8

---

 WHITE PAPER ACHIEVING ORDER THROUGH CHAOS  ENGINEERING: A SMARTER WAY TO  BUILD SYSTEM RESILIENCE  {{ img-description : a green plant growing through dirt and rocks, in the style of explosive pigmentation, 500–1000 ce, rasquache, softly organic, organic, malick sidibé, marguerite blasingame -- }} Abstract Digital infrastructure has grown increasingly complex owing to distributed  cloud architectures and microservices. More than ever before, it is  increasingly challenging for organizations to predict potential failures and  system vulnerabilities. This is a critical capability needed to avoid expensive  outages and reputational damage.  This paper examines how chaos engineering helps organizations boost  their digital immunity. As a leading quality engineering approach, chaos  engineering provides a systematic, analytics-based, test-first, and well- executed path to ensuring system reliability and resilience in today’s  disruptive digital era.

---

  Page: 2 / 8

---

  Introduction Digital systems have become increasingly complex and  interdependent, leading to greater vulnerabilities across  distributed networks. There have been several instances where  a sudden increase in online traffic or unforeseen cyberattacks  have caused service failures, adversely impacting organizational  reputation, brand integrity, and customer confidence. Such  outages have a costly domino effect, resulting in revenue losses or,  in some cases, regulatory action against the organization.  Thus, enterprises must implement robust and resilient quality  engineering solutions that safeguard them from potential  threats and help overcome these challenges. This is where ‘chaos  engineering’ comes in.<|endoftext|>Chaos engineering is a preventive measure that tests failure  scenarios before they have a chance to grow and cause  downtime in live environments. It identifies and fixes issues  immediately by recognizing system weaknesses and how  systems behave during an injected failure. Through chaos  engineering, organizations can establish mitigation steps to  safeguard end users from negative impact and build confidence  in the system capacity to withstand highly variable and  destructive conditions.<|endoftext|>Chaos Engineering – A Boost to Digital  Immunity System resilience is about how promptly a system can recover  from disruption. Chaos engineering is an experimentative  process that deliberately disrupts the system to identify weak  spots, anticipate failures, predict user experience, and rectify the  architecture. It helps engineering teams redesign and restore  the organization’s infrastructure and make it more resilient  in the face of any crisis. Thus, it builds confidence in system  resiliency by running failure experiments to generate random and  unpredictable behaviour.  Despite its name, chaos engineering is far from chaotic. It is a  systematic, data-driven technique of conducting experiments  that use chaotic behaviour to stress systems, identify flaws, and  demonstrate resilience. System complexity and rising consumer  expectations are two of the biggest forces behind chaos  engineering. As systems becoming increasingly feature-rich,  changes in system performance affect system predictability and  service outcomes, which in turn, impact business success.<|endoftext|>{{ img-description :  a group of hands putting pieces of wooden gears together, in the style of bold structural designs, meticulous detailing (pin background) }}

---

  Page: 3 / 8

---

  How Chaos Engineering is Different from  Traditional Testing Practices • Performance testing – It baselines application performance  under a defined load in favorable environmental conditions.  The main objective is to check how the system performs when  the application is up and running without any severe functional  defects in an environment comparable to the production  environment. The potential disruptors uncovered during the  performance tests are due to certain load conditions on the  application.  • Disaster recovery testing – This process ensures that an  organization can restore its data and applications to continue  operations even after critical IT failure or complete service  disruption.<|endoftext|>• Chaos testing – During the chaos test, the application  under normal load is subjected to known failures outside the  prescribed boundaries with minimum blast radius to check if the  system behaves as expected. Any deviation from expectations  is noted as an observation and mitigation steps are prepared to  rectify the deviation.<|endoftext|>Quality assurance engineers find chaos testing to be more effective  than performance and disaster recovery testing in unearthing  latent bugs and identifying unanticipated system weaknesses.<|endoftext|>{{ img-description : a group of red jet planes flying through the sky, in the style of keos masons, expert draftsmanship, photo taken with provia, performance, wimmelbilder, grid, anglocore  (pin background) }}

---

  Page: 4 / 8

---

  5-step Chaos Engineering Framework Much like a controlled injection, implementing chaos engineering  calls for a systematic approach. The five-step framework described  below, when ‘injected’ into an organization, can handle defects  and fight system vulnerabilities.  Chaos engineering gives organizations a safety net by introducing  failures in the pre-production environment, thereby promoting  organizational learning, increasing reliability, and improving  understanding of complex system dependencies.     1. Prepare the process   Understand the end-to-end application architecture. Inform  stakeholders and get their approval to implement chaos  engineering. Finalize the hypothesis based on system  understanding.<|endoftext|>2. Set up tools   Set up and enable chaos test tools on servers to run chaos  experiments. Enable system monitoring and alerting tools. Use  performance test tools to generate a steady load on the system  under attack. Additionally, a Jenkins CI/CD pipeline can be set  up to automate chaos tests.<|endoftext|>3. Run chaos tests  Orchestrate different kinds of attacks on the system to cause  failures. Ensure proper alerts are generated for the failures and  sent to the right teams to take relevant actions.<|endoftext|>4. Analyze the results  Analyze the test results and compare these with the  expectations set when designing the hypothesis. Communicate  the findings to the relevant stakeholders to make system  improvements.<|endoftext|>5. Run regression tests   Repeat the tests once the issues are fixed and increase the blast  radius to uncover further failures.<|endoftext|>This step-by-step approach executes an attack plan within the  test environment and applies the lessons/feedback from the  outcomes, thereby improving the quality of production systems  and delivering tangible value to enterprises.<|endoftext|>{{ img-description : three workers with laptops looking at their company's finances, in the style of sabattier filter, frédéric fiebig, photo taken with provia, back button focus, aquirax uno, handheld, interactive  }}

---

  Page: 5 / 8

---

  Examples of Chaos Engineering Experiments A chaos engineering experiment or a chaos engineering attack is  the process of inducing attacks on a system under an expected  load. An attack involves injecting failures into a system in a simple,  safe, and secure way.  There are various types of attacks that can be run against  infrastructure. This includes anything that impacts system  resources, delays or drops network traffic, shuts down hosts, and  more. A typical web application architecture can have four types  of attacks run on it to assess application behavior: • Resource attacks – Resource attacks reveal how an application  service degrades when starved of resources like CPU, memory,  I/O, or disk space • State attacks – State attacks introduce chaos into the  infrastructure to check whether the application service fails or  whether it handles it and how  • Network attacks – Network attacks demonstrate the impact of  lost or delayed traffic on the application. It is done to test how  services behave when they are unable to reach any one of the  dependencies, whether internal or external • Application attacks – Application attacks introduce sudden  user traffic on the application or on a particular function. It is  done to test how services behave when there is sudden rise in  the user traffic due to high demand.<|endoftext|>{{ img-description:two men standing together at a computer in a factory, in the style of lively tableaus, innovating techniques, precise craftsmanship, spot metering, heidelberg school, dynamic brushwork vibrations, curved mirrors  }}

---

  Page: 6 / 8

---

  GameDay Concept {{ img-description : people standing around table while looking at notes, in the style of complex layering, blueprint }} GameDay is an advanced concept of chaos engineering. It is  organized by the chaos test team to practice chaos experiments,  test incident response process, validate past outages, and find  unknown issues in services.  In GameDay, a mock war room is set up and the calendar of all  stakeholders is blocked for up to 2-4 hours. One or more chaos  experiments are run on the system or service to observe the  Figure 2 – GameDay simulation approach impact. All technical outcomes are discussed.<|endoftext|>The team includes a ‘General’ who is responsible for conducting  the GameDay, a ‘Commander’ who coordinates with all the  participants, ‘Observers’ who monitor the GameDay tests and  validate the deviations (if any), and a ‘S