An Oracle White Paper March Load Testing Best Practices for Oracle E- Business Suite using Oracle Application Testing Suite - PDF

Please download to get full document.

View again

of 21
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

How To, Education & Training

Published:

Views: 6 | Pages: 21

Extension: PDF | Download: 0

Share
Related documents
Description
An Oracle White Paper March 2013 Load Testing Best Practices for Oracle E- Business Suite using Oracle Application Testing Suite Executive Overview... 1 Introduction... 1 Oracle Load Testing Setup... 2
Transcript
An Oracle White Paper March 2013 Load Testing Best Practices for Oracle E- Business Suite using Oracle Application Testing Suite Executive Overview... 1 Introduction... 1 Oracle Load Testing Setup... 2 OLT Server... 2 OLT Database... 3 OLT Agent... 3 Hardware Estimation... 4 Hardware Specifications... 5 Oracle E-Business Suite Application Setup... 6 Method... 7 Oracle E-Business Suite Transactions... 8 Scenario Configuration... 9 Monitoring Server Performance... 9 Results Throughput Server Response Time Hits per Second Graphs Performance Graphs Oracle EBS Application Servers CPU & Memory Utilization Summary... 18 Executive Overview Oracle s Product Development IT (Oracle PDIT) division oversees massive data stores, supporting the development of products and services. Oracle PDIT s infrastructure empowers more than 21,000 software engineers to develop Oracle database, middleware, and application products on a daily basis. They also run all the internal back-office business applications, oracle.com, Oracle Self Service, Oracle Support Services, and company and collaborative tools. Oracle PDIT uses Oracle Application Testing Suite software to run performance, scalability, and stress tests to ensure server hardware and software can scale to support its global user base of 100,000 users. This white paper describes how Oracle PDIT is doing stress testing with Oracle Load Testing, a component of Application Testing Suite, and provides best practices to ensure successful load testing for Oracle E-Business Suite applications. Introduction Oracle Application Testing Suite is an integrated, full lifecycle solution which ensures application quality and performance with complete end-to-end testing and test management capabilities. Oracle Application Testing Suite helps deliver high quality applications with three separately licensed products: Oracle Functional Testing for automated functional and regression testing of Web applications, Web Services, Oracle packaged applications and databases. Oracle Load Testing for automated load and performance testing of Web applications, Web Services, Oracle packaged applications and databases. Oracle Test Manager for process management throughout the testing lifecycle, including test planning, requirements management, test management, test execution and defect tracking. This white paper focuses on Oracle Load Testing component of Application Testing Suite. In it we will describe the benchmark results of Oracle PDIT running 10,588 and 21,195 concurrent users on Oracle s E-Business Suite (EBS) Applications Release using Oracle Load Testing Oracle Load Testing Setup Oracle Load Testing (OLT) has three main components: Server, Agent, and Database. The OLT Server has a Controller module which connects to the Agent and sends execution information to simulate users for the load test. OLT Server also has a ServerStats module which creates a configuration in order to monitor data from various tiers of the Application under Test. The data will be collected by the data collector component residing on the Agent, based on the monitoring configuration created by ServerStats. The data collected is presented via meaningful graphs and reports which allow the user to quickly determine the bottleneck in their application stack. The OLT Database is used to store scenario configuration and load test results for real-time and post-run reporting. Figure 1. Oracle Load Testing Architecture OLT Server The OLT Server has two modules. OLT Controller is the conductor of the load test, it orchestrates how a load test is performed. While the load test is running, the data collector component collects relevant data which aids in the detection of bottlenecks and key performance inhibitors based on the monitors configured by OLT ServerStats. OLT ServerStats is provided as an out of the box tool to monitor the Application Under Test s performance. However, users can use other monitoring tools such as Oracle Enterprise Manager Grid Control or the application s built in monitoring capabilities in parallel or standalone. The benefit of using OLT ServerStats simultaneously during a load test is that the reports will be made available through OLT, and can leverage OLT Controller s reporting engine to generate custom reports and graphs for reporting purposes. 2 The OLT Server is the most critical component of a load test system and should have enough resources to support the test. The following best practices are recommended for an OLT Server system: Select a test system with the most resources available (CPU & memory) to run the OLT Server. OLT Server requires a minimum of 2GB RAM to operate, however this is only enough to run a session for demonstration and evaluation purpose. Typically in a production load test, a system with more than 8GB of free RAM available is used to run the OLT Server. OLT Database Oracle Application Testing Suite (ATS) is shipped with Oracle Database 10g Express Edition (XE) and this is the default OLT database configured on installation. Oracle XE for OLT database is useful for demonstration or product evaluation as it has a limit of 4GB for user data storage. For production load testing it is recommended to use Oracle Database 11g Enterprise Edition. The following best practices are recommended for an OLT Database system: Select Oracle Database 11g Enterprise Edition database for production load testing. ATS comes with a restricted use license of Oracle Database 11g for all versions. Prepare a separate and dedicated system to host the OLT Database. The OLT Database should be physically located near the OLT Controller in order to reduce latency. Users can configure new databases using the ATS Database Configuration tool. Monitor the resource utilization of the database during the load test to ensure it runs in a healthy state and does not become the bottleneck. OLT Agent OLT Agent is the orchestrator of the load test. It simulates user load via Virtual Users according to the directions of the OLT Controller. The following best practices are recommended for OLT Agent systems: Run the OLT Agents on dedicated systems separate from the OLT Server. Technically OLT Controller can run Virtual Users on the same machine; however that is not recommended for a production load test as the Agent resource utilization can affect OLT Server's operation when both Controller and Agent are residing on the same system. Ensure that the OLT Agent systems and OLT Server systems are running the same ATS version and build number. Agent initialization will fail if the version and build number is different from that of the OLT Server. Size the amount of Virtual Users to run on a given OLT Agent system based on the amount of memory available on that system. It is recommended to benchmark the Agent systems to determine how many Virtual Users they can run prior to running your actual load test. This can be done by running a single Virtual User on an Agent system then incrementally adding one Virtual User at a 3 time, and monitoring the additional memory and CPU consumed for each additional Virtual User. More detail is provided in the Hardware Estimation section of this page. Monitor the throughput between the OLT Agent systems and the Application Under Test (AUT) to make sure the bandwidth limitations are not exceeded. Because each Virtual User is making requests, even if the Agent systems have enough resources to run the target number of Virtual Users, it could still reach a bandwidth limitation as the amount of data that goes between the Agent system and AUT increases. Hardware Estimation In order for a load test to be successful, it is essential to install OLT components on appropriately sized hardware. As part of load test planning, one should go through the exercise of hardware estimation to determine the number of OLT Agent systems required for the load test. Hardware sizing depends on the script type, scenario setting, and environment. In Oracle Load Testing 12.2, a new Hardware Estimation tool was introduced which provides an estimate of the hardware requirements for a given scenario. The Hardware Estimation tool can generate a report to determine how much hardware will likely be required to run the specified load test scenario, and how to modify the session configurations to run more Virtual Users on one machine. The following script types are supported for hardware estimation: Web/HTTP, Oracle EBS/Forms, Oracle Fusion/ADF, and Oracle Siebel. OLT 12.1 was used for PDIT s testing because 12.2 was not available at the time. Due to this, the memory requirement (JVM heap) per Virtual User was determined manually. There are two manual configuration methods to measure the memory requirements. 1. JVM Monitoring Tools 2. Benchmark Agent Systems JVM Monitoring Tools Oracle JRockit Mission Control is a JVM profiling tool that can be used to monitor agent process heap and process utilization. This can be done by using the JConsole tool as well. In order to use JRockit Mission Control, monitor the agent processes by running the OLT scenario and incrementing the load by a small number of Virtual Users at a time (i.e. 10 Vusers every 5 minutes). A 100 Virtual User load test will provide a good sample of the resource consumption pattern for the load test. Extrapolate the results from this test to the real load test requirements to determine the number of OLT Agent systems required. Benchmark Agent Systems This is the method used by Oracle PDIT. A small heap size (1 GB) was configured for each script in the load test scenario and one Virtual User was run with memory footprint on the agent system noted. Additional Virtual Users were added incrementally with memory footprints noted as well. This process was repeated until the memory maxed out on the OLT Agent system. The benefit of using this method 4 is its handiness. It does not require any additional installation or knowledge of other tools, and can generate results quickly for hardware estimation. In the case of Oracle PDIT, the test session was terminated by out of memory error when it reached 500 Virtual Users. Since the Max JVM Heap Size set for each script was 1 GB, the following calculation was made: 1 GB = 1024 MB 1024 MB / 500 Virtual Users = MB RAM required per Virtual User 1 Hardware Specifications Based on the hardware estimation exercise above, PDIT calculated 2MB of RAM was required per Virtual User. The required memory footprint was 24 GB for 12,000 Virtual Users and 48 GB for 24,000 Virtual Users. PDIT had access to five OLT Agent systems and decided to distribute the load across all five agents. Below is their OLT Controller and Maximum JVM Heap Size setting. OLT Controller JVM Heap Size: 3 GB 2 Maximum JVM Heap Size (MB) for each script: The following are hardware specifications for PDIT s OLT Server and Agent systems. OLT Server (x1) Sun Fire 4170 M2 Windows Server 2008 R2 64-bit OS Intel Xeon X5670 dual processor (2.93 GHz each), 96 GB RAM OLT Agents (x5) Sun Fire 4150 Oracle Enterprise Linux Server release 5.5 (Carthage) GHz, 60 GB RAM MB RAM per Virtual User can be used as a very rough guideline for Agent hardware. This value may change based on the script and is no substitute for taking a real measurement from the real script(s) being tested. 2 3 GB is the maximum JVM heap size that can be configured on the OLT Controller (which is installed on a 64-bit Windows system) because ATS ships with a 32-bit version of JRockit JVM. 3 The 50 GB RAM allocated for JVM heap for the load test is more than sufficient to meet the minimum requirements. 5 Oracle E-Business Suite Application Setup Table 1 below describes the hardware and software specifications of the Oracle E-Business Suite application server used for Oracle PDIT s load tests. TABLE 1. ORACLE E-BUSINESS SUITE APPLICATION SETUP ENVIRONMENT COMPONENT CONFIGURATIONS Middle Tier Servers Self Service Application Server 1 Model : X4170_M2 CPU Num : 8 RAM : 94 GB Operating System : Enterprise Linux Rel. 5.5 (Carthage) Self Service Application Server 2 Model : X4170_M2 CPU Num : 8 RAM : 96 GB Operating System : Enterprise Linux Rel. 5.7 (Carthage) Forms Server 1 Model : X4170_M2 CPU Num : 6 RAM : 70 GB Operating System : Enterprise Linux Rel. 5.5 (Carthage) Forms Server 2 Model : X4170_M2 CPU Num : 8 RAM : 96 GB Operating System : Enterprise Linux Rel. 5.7 (Carthage) Forms Server 3 Model : X4170_M2 CPU Num : 6 RAM : 70 GB Operating System : Enterprise Linux Rel. 5.7 (Carthage) irecruitment (irec) Server Model : X4170_M2 CPU Num : 13 RAM : 32 GB Operating System : Enterprise Linux Rel 5.5 (Carthage) Concurrent Manager Server 1 Model : X4170_M2 CPU Num : 4 RAM : 48GB Operating System : Enterprise Linux Rel. 5.5 (Carthage) Concurrent Manager Server 2 Model : X4170_M2 CPU Num : 8 RAM : 96 GB Operating System : Enterprise Linux Rel. 5.7 (Carthage) Database Servers Model : SPARC SuperCluster T4-4 (4 nodes *) * Each node has 1TB of RAM * Each node has 32 CPU's, each of which is 8-way hyper threaded, for a total of 256 threads per node Operating System : SunOS 5.11 Storage Device : Oracle Exadata Storage Server 6 Method The Oracle EBS production database was monitored during a typical quarter end close and the average number of connections made during this time was 12,000. Two separate load tests were performed by PDIT. The first load test simulated 12,000 Virtual Users as a real life production load during Oracle s quarter end and established baseline results. Following the baseline test, a 2x production load test was run to see if the hardware and software configuration can support twice the amount of production workload with reserve capacity. These tests will generate different mixes and levels of application activity and exercise different infrastructure components with a special emphasis to study application and platform activity, resource utilization levels, and potential contention. TABLE 2. ORACLE PDIT LOAD TEST METHODOLOGY TEST PURPOSE GOAL ACTUAL NUMBER OF VIRTUAL USERS 1 1x Quarter End 1) Simulate a real life production load of 12,000 Virtual Users as a baseline test. 2) Identify regressions, concurrency issues, application and database errors due to hardware & software configuration changes. 3) Assess the server response time, performance, server CPU & memory utilization as a baseline. 10,588 2x Quarter End 1) Simulate a production load of 24,000 Virtual Users to see if the application can scale to meet this target. 2) Identify regressions, concurrency issues, application and database errors due to hardware & software configuration changes. 3) Compare server response time, performance, server CPU & memory utilization between 1x Quarter End and 2x Quarter End load tests. 21,195 1 The actual number of Virtual Users for the 1x and 2x Quarter End load tests are different than the stated goal because one Forms session will create multiple sessions against the database. While the target remained at 12,000 and 24,000 database sessions, less Forms users were required in order to achieve that. For the 1x Quarter End load test, there was a 1.68% error rate (181 Virtual Users failed out of 10,769 running Virtual Users). For the 2x Quarter End load test there was a 0.34% error rate (73 Virtual Users failed out of 21,268 running Virtual Users). 7 Oracle E-Business Suite Transactions Test scripts are based on E-Business Suite user requirements and exercise some of the most critical and frequently used components of the application. The test scripts must run in a steady state for at least one hour in order to meet the requirements of the performance test. These tests simulate the top 15 business transactions on Oracle s Self Service application (E-Business Suite Applications Release ). The business transactions are a mix of web only (Self Service) and web/forms EBS components. TABLE 3. ORACLE E-BUSINESS SUITE TRANSACTIONS LOAD TEST SCRIPT TECH STACK 1X QUARTER END CONCURRENT USER SESSIONS 2X QUARTER END CONCURRENT USER SESSIONS Enter Time Card Forms View Concurrent Request Output Forms irec Search Vacancy Self Service View Pay Slips Self Service Internal irecruitment Search Self Service FS Advance Exchange Copy Request Forms View Image Transaction Self Service Vacation Inquiry Self Service Project List Search Self Service Contracts Check QA Forms Purchase Requisition Entry Self Service Enter Cancel Order Forms OKS Show All Products Forms OKC Billing Schedule Forms US Expense Reporting Self Service Scenario Configuration Default scenario configuration settings were used for each load test with the exception of the settings listed in Table 4. These settings were customized in order to accommodate for the large number of Virtual Users run by PDIT's load tests. As best practice, use a non-zero iteration delay and realistic user think times. For the socket and request timeout settings, a higher response threshold than the default 120 seconds was set as PDIT wanted to allow more time in case a long-running operation took longer than 120 seconds. It is also recommended to keep the Max JVM Heap size the same across all scripts for a particular agent system in order to avoid spawning multiple Agent processes. TABLE 4. ORACLE PDIT LOAD TEST CUSTOM SCENARIO CONFIGURATIONS SETTING 1X QUARTER END VALUE 2X QUARTER END VALUE Iteration Delay (Seconds) 150 to to 488 Think Time (Seconds) Recorded think time (0 10) Recorded think time (0 10) Virtual User (VU) Ramp-Up Add 15 users every 5 seconds Add 15 users every 5 seconds Scenario Duration 60 minutes of steady state 60 minutes of steady state Number of Users 10,769 21,268 Number of OLT Agents 5 5 Socket Timeout (Seconds) Request Timeout (Seconds) Connection Idle Timeout (Seconds) Maximum JVM Heap Size (MB) Monitoring Server Performance OLT includes a ServerStats module which is a built-in server monitoring capability. Users can select to use ServerStats or other monitoring tools to collect performance metrics from the application's infrastructure under load. PDIT used Oracle Enterprise Manager Grid Control to capture the server CPU, Memory, and Swap Space on DB and Mid Tier (Forms, SSA, and irec) during the load tests. 9 Results Oracle PDIT's load test results are discussed in this section. Throughput Throughput levels achieved from PDIT s load tests are shown in Figures 2 4. The 1x Quarter End target throughput was calculated based on the average production load during a typical financial quarter end close. For the 2x Quarter End load test, the targets were calculated by multiplying 1x Quarter End targets by 2, as the number of Virtual Users for the load test was doubled. Figure 2 shows the target number of active Virtual Users is 10,769 for the 1x Quarter End load test, and 21,268 for the 2x Quarter End load test. The average number of running Virtual Users was 10,508 and 20,974, respectively, which was very close to the established target. There was a small number of failed Virtual Users for both load tests, which is common and expected for a load test. Figure 2. Load Test Throughput Results Active Virtual Users Transactions per second is the number of times the Virtual User played back the script per second. Figure 3 shows the target number of transactions per second is 30 for the 1x Quarter End load test, and 60 for the 2x Quarter End load test. The average number of transactions per second was 40 and 64, respectively, which was more than the target. Oracle Load Testing generated a slightly higher load against the server, which is acceptable as the minimum target has been met. 10 Figure 3. Load Test Throughput Results Transactions Per Second Hits per second is the number of resource requests to the server per second. Figure 4 shows the target number of hits per second is 1000 for the 1x Quarter End load test, and 2000 for the 2x Quarter End load test. The average number of hits per second was 1363 and 2177, respectively, which was higher than the stated target. This is in line with the transa
Recommended
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x