An Oracle White Paper November Oracle Exalytics Deployment for High Availability - PDF

Please download to get full document.

View again

of 26
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

Magazines/Newspapers

Published:

Views: 8 | Pages: 26

Extension: PDF | Download: 0

Share
Related documents
Description
An Oracle White Paper November 2014 Oracle Exalytics Deployment for High Availability Introduction... 1 Deploying Exalytics for High Availability... 2 Web Tier... 5 Application Tier... 5 Data Tier... 5
Transcript
An Oracle White Paper November 2014 Oracle Exalytics Deployment for High Availability Introduction... 1 Deploying Exalytics for High Availability... 2 Web Tier... 5 Application Tier... 5 Data Tier... 5 Exalytics Prerequisites for Deployment... 5 Exalytics and Exadata InfiniBand Configuration... 6 Exalytics Deployment on the First Exalytics Node... 9 Post Installation Steps Oracle WebLogic Server Oracle Business Intelligence Basic Exalytics Start-Stop Scripts first Node Exalytics Deployment on the Second Exalytics Node Oracle TimesTen Installation Oracle WebLogic Server Installation Oracle Business Intelligence Installation Basic Exalytics Start-Stop Scripts Second Node Building Aggregates Create Aggregates in the Local TimesTen Instance Create Aggregates in the Remote TimesTen Instance Refresh Aggregates in the Local TimesTen instance Refresh Aggregates in the Remote TimesTen instance Additional Considerations... 23 Introduction Oracle Exalytics In-Memory Machine provides the fastest performance for business intelligence applications. These applications often provide critical information to business decision-makers and that requires the application to be available. To ensure the applications are available when they re needed, a high availability configuration is required to eliminate the single points of failure that can result in an application outage. This white paper contains technical details about how to configure a two node cluster of Oracle Exalytics systems running Oracle Business Intelligence Enterprise Edition with Oracle Exadata to enable high availability. The Maximum Availability Architecture (MAA) configuration requires horizontal scale out to be configured for scalability and performance (load balancing). 1 Deploying Exalytics for High Availability Oracle Exalytics runs the Oracle Business Intelligence (BI) Foundation along with Oracle TimesTen In-Memory Database. Oracle Exalytics supports both active-active and activepassive clustering to provide scalability and high availability. This MAA white paper will focus on the Oracle Exalytics active-active cluster configuration, where each Exalytics machine in the cluster runs its own Oracle BI Foundation and Oracle TimesTen database. Each BI Server has a separate Data Source Name (DSN) for communication with each TimesTen instance. Shared storage such as Oracle ZFS Appliance is configured for BI server repository, BI presentation catalog, and Global Cache. Note: BI local cache resides on the local node on RAM disk. Each Exalytics machine has InfiniBand connectivity to the Oracle Exadata Database Machine running the application database to take advantage of flash technology, cell offload processing, high availability, and other optimizations provided by Exadata. Oracle HTTP servers must be installed on a separate pair of physical or virtual hosts between the load balancer and Exalytics machines. 2 Figure 1: The full set of components in a High Availability cluster 3 Exalytics1.mycompany.com Exalytics2.mycompany.com Web Logic EE Presentation Server BI Server RPD Logical Table Source 3) Disk Oriented DB 2) Remote TT IMDB 1) Local TT IMDB odbc.ini TimesTen sys.odbc.ini sys.ttconnect.ini IMDB Shared File Server Shared webcat Dashboards Reports BI Config files Global Cache RPD Shared DB Server RAC Web Logic EE Presentation Server BI Server RPD Logical Table Source 3) Disk Oriented DB 2) Remote TT IMDB 1) Local TT IMDB odbc.ini TimesTen sys.odbc.ini sys.ttconnect.ini IMDB Temp Perm Perm Temp Figure 2: High Availability configuration elements for OBI EE and TimesTen The two environments are effectively identical other than their host name, so the configuration of the sys.odbc.ini (TimesTen) and odbc.ini (OBI EE) files are identical. These both reference a TimesTen data store that is local to them and one that is remote to them. The difference is in the sys.ttconnect.ini file where the network addresses of the physical hosts are defined. The BI Server will select the Logical Table Source (LTS) with the largest grain (the most aggregated values) that can satisfy any given query. This results in queries to the fact table with the smallest number of rows. If it finds two LTSes at the same grain, then it will prioritize the first one it finds in the list. Figure 2 uses numbers to show the relative order that the LTSes should appear in the RPD to reduce line collisions. In the RPD the aggregate LTSes would appear in the reverse order 4 Web Tier In the web tier, two nodes, WEBHOST1 and WEBHOST2, run Oracle HTTP Server configured with mod_wl_ohs. Through mod_wl_ohs, which allows requests to be proxied from Oracle HTTP Server to Oracle WebLogic Server, Oracle HTTP Server forwards requests to Oracle WebLogic Server running in the application tier. The web tier should include an external load balancer to handle external requests. External requests are sent to the virtual host names configured on the load balancer. The load balancer then forwards the requests to the Oracle HTTP Server. Application Tier Nodes in the application tier are located in the DMZ secure zone. In this tier, Oracle WebLogic Server is configured with Managed Servers for running Oracle BI Foundation and the Oracle TimesTen In-Memory Database. Multiple Exalytics servers are clustered together using InfiniBand for high availability. For more information, see Oracle Exalytics In-Memory Machine Owner's Guide, Chapter 7 Connecting Oracle Exalytics In- Memory Machine to Oracle Exadata Database Machine and Chapter Scenario 2: Clustered Exalytics In-Memory Machine with Oracle Exadata Database Machine. Note: Due to lack of InfiniBand support in the Firewalls today Exalytics placement in the Application Tier makes only sense if using Ethernet connectivity between Exalytics and Exadata. Until InfiniBand Support is available in Firewalls we recommend Exalytics to be located in the Data Tier. Data Tier Nodes in the data tier are located in the most secured network zone (the intranet). In the data tier, the Oracle Exalytics machines are connected to Oracle Exadata Database Machine over InfiniBand. Exalytics Prerequisites for Deployment 1. Ensure that the prerequisites in Oracle Fusion Middleware Installation and Administration Guide for Oracle Exalytics In-Memory Machine for Installing Software on Exalytics are met. 2. Connect Exalytics HCA ports to each Exadata Database Machine InfiniBand leaf switch. One of ports 5B, 6A, 6B, 7A, 7B, or 12A on the leaf switches of the Exadata Database Machine should be used even if additional ports are open. On partial rack configurations, some ports are reserved for potential future expansion. 5 3. Configure DNS client settings on the Exalytics machine. It is best to use the same DNS servers as used by the Exadata Database Machine nodes if possible. 4. Enable NTP client settings on the Exalytics machine. It is best to use the same NTP servers as used by the Exadata Database Machine nodes if possible. 5. Configure HugePages if TimesTen requires more than 256 GB. Activate the number of HugePages in /etc/sysctl.conf file and by setting the parameter vm.nr_hugepages = The largest recommended memory for TimesTen is a number that leaves enough memory for the other installed components and the Operating System to function adequately; the larger the number of concurrent users the higher this memory value needs to be. As a very rough guide no more than 80% - 85% of the memory should be devoted to active TimesTen databases. If you are running EPM on the same Exalytics machine then that number should be reduced to provide adequate memory resources to the EPM system. Exalytics and Exadata InfiniBand Configuration 1. Enable automatic RDS kernel module loading on Exalytics. This can be done in rolling fashion a. Edit /etc/infiniband/openib.conf to set RDS_LOAD=yes and SDP_LOAD=yes. b. Edit the /etc/modprobe.conf file and add this at the end as single line: options ib_sdp sdp_zcopy_thresh=0 recv_poll=0 sdp_apm_enable=0 Note: In Exalytics physical environment, disable APM at Exalytics node level. In Exalytics virtual environment, disable APM at guest VM (DomU) level. The setting sdp_apm_enable=0 is if Exadata Image is x or x and if this parameter is defined in the Linux kernel module ib_sdp modinfo ib_sdp grep sdp_apm_enable c. Reboot after change. 2. Create InfiniBand listener on each Exadata compute nodes. This can be done in rolling fashion a. Edit /etc/infiniband/openib.conf to set RDS_LOAD=yes and SDP_LOAD=yes. b. Edit /etc/ofed/libsdp.conf and change both lines. Change use both server * : use both client * : 6 to use tcp server * *:* use tcp client * *:* use tcp means that TCP is the default protocol, SDP is available on demand. c. Edit the /etc/modprobe.conf file and add this at the end as a single line: options ib_sdp sdp_zcopy_thresh=0 recv_poll=0 sdp_apm_enable=0 The setting sdp_apm_enable=0 is if Exadata Image is x or x and if this parameter is defined in the Linux kernel module ib_sdp modinfo ib_sdp grep sdp_apm_enable d. Reboot the compute node. e. Edit /etc/hosts on each Exalytics machine and Exadata Database Machine compute node to add the virtual IP addresses for use on the InfiniBand network, for example, Exadata Database Machine quarter rack (2 compute nodes). Choose a valid IP address on the IB private network for each Exadata compute node and ensure those IPs are not in use currently dm01db01-ibvip.mycompany.com dm01db01-ibvip dm01db02-ibvip.mycompany.com dm01db02-ibvip f. As root user on one of the Exadata Database Machine compute nodes, execute the following steps to create a network resource for the InfiniBand network, and add start virtual IP addresses for each compute node on it. Make appropriate substitutions for the network and subnet mask in use on your private network (it may not be / ). Ask your administrator or use ipcalc, based on network mask to calculate the network. srvctl add network -k 2 -S / /bondib0 srvctl add vip -n dm01db01 -A dm01db01-ibvip/ /bondib0 -k 2 srvctl add vip -n dm01db02 -A dm01db02-ibvip/ /bondib0 -k 2 Starting with X4 hardware using Active/Active IB, use instead: srvctl add network -k 2 -S / /ib0\ ib1 srvctl add vip -n dm01db01 -A dm01db01-ibvip/ /ib0\ ib1 -k 2 srvctl add vip -n dm01db02 -A dm01db02-ibvip/ /ib0\ ib1 -k 2 7 Bug fix in GI HOME is required to bring up new cluster ware resources. It is included in BP21, , and srvctl start vip -i dbm01db01-ibvip srvctl start vip -i dbm01db02-ibvip g. On one of the Exadata Database Machine compute nodes, as oracle user (or owner of the Grid Infrastructure ORACLE_HOME), execute following command srvctl add listener -l LISTENER_IB -k 2 -p TCP:1522/SDP:1522 h. On each Exadata Database Machine compute node, as oracle user (or owner of the Grid Infrastructure ORACLE_HOME), create or append to tnsnames.ora in GRID_HOME/network/admin the following new entries Note: *IBREMOTE and *IBLOCAL entries will be slightly different on each node to reference the proper remote and local VIP names-you cannot copy the entries from one node to another without some modifications: DBM = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = dm01- scan.mycompany.com)(port = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = dbm) )) DBM_IB = (DESCRIPTION = (LOAD_BALANCE=on) (ADDRESS = (PROTOCOL = TCP)(HOST = dm01db01- ibvip.mycompany.com)(port = 1522)) (ADDRESS = (PROTOCOL = TCP)(HOST = dm01db02- ibvip.mycompany.com)(port = 1522)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = dbm) )) LISTENER_IBREMOTE = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = dm01db02- ibvip.mycompany.com)(port = 1522)) )) LISTENER_IBLOCAL = 8 (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = dm01db01- ibvip.mycompany.com)(port = 1522)) (ADDRESS = (PROTOCOL = SDP)(HOST = dm01db01- ibvip.mycompany.com)(port = 1522)) )) LISTENER_IPLOCAL = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = dm0101- vip.mycompany.com)(port = 1521)) )) LISTENER_IPREMOTE = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = dm01- scan.mycompany.com)(port = 1521)) )) i. On one Exadata Database Machine compute node, connect to the database instance as sysdba. SQL alter system set listener_networks='((name=network2) (LOCAL_LISTENER=LISTENER_IBLOCAL)(REMOTE_LISTENER=LIST ENER_IBREMOTE))', '((NAME=network1)(LOCAL_LISTENER=LISTENER_IPLOCAL)(REMOTE _LISTENER=LISTENER_IPREMOTE))' scope=both; j. On one Exadata Database Machine compute node, as oracle user (or owner of the Grid Infrastructure ORACLE_HOME) restart the listener named LISTENER_IB. srvctl stop listener -l LISTENER_IB srvctl start listener -l LISTENER_IB Exalytics Deployment on the First Exalytics Node Follow Section Deploying Oracle Exalytics for High Availability in Oracle Fusion Middleware Installation and Administration Guide for Oracle Exalytics In-Memory Machine. In the input.properties file specify the virtual IP addresses you used for the InfiniBand network. For example: 9 DATABASE_CONNECTION_STRING= dbm01db01-ibvip.mycompany.com:1522:dbm1^dbm01db02- Respectively in RCU Connect String: dbm01db01-ibvip.mycompany.com:1522:dbm1^dbm01db02- Post Installation Steps Oracle TimesTen 1. Run the daemon configuration scripts as root to start TimesTen on boot. /u01/app/oracle/product/timesten/tt1122/bin/setuproot -install 2. Configure large pages. You can configure large pages for the TimesTen database by editing the following file and adding the following lines to /u01/app/oracle/product/timesten/tt1122/info/ttendaemon.options -linuxlargepagealignment 2 3. Define the logical Local and Remote server name for Oracle TimesTen in /u01/app/oracle/product/timesten/tt1122/info/sys.ttconnect.ini [tt_exalytics_local] Description=TimesTen Server Network_Address=exalytics1.mycompany.com TCP_PORT=53397 [tt_exalytics_remote] Description=TimesTen Server Network_Address=exalytics2.mycompany.com TCP_PORT= Define the Local and Remote TimesTen In-Memory Database using a DSN that is contained in the /u01/app/oracle/product/timesten/tt1122/info/sys.odbc.ini file. [ODBC Data Sources] TT_AGGR_STORE_LOCAL=TimesTen Driver TT_AGGR_STORE_REMOTE=TimesTen Client Driver # The DSN below defines a TimesTen In-Memory Database with 40GB devoted to # tables and indexes and 40GB for handling queries and their results, 10 # the overall memory consumption will be approximately 81GB (PermSize + TempSize + LogBufMB + 64). [TT_AGGR_STORE_LOCAL] Driver=/u01/app/oracle/product/fmw/../TimesTen/tt1122/lib/libtten.so DataStore=/u01/app/oracle/product/fmw/../aggregate_store/tt_aggr_store LogDir=/u01/app/oracle/product/fmw/../aggregate_store/logs DatabaseCharacterSet=AL32UTF8 ConnectionCharacterSet=AL32UTF8 LogFileSize=1024 LogBufMB=1024 LogBufParallelism=16 Preallocate=0 PermSize=40000 TempSize=40000 MemoryLock=4 CkptFrequency=30 CkptLogVolume=0 CkptRate=0 PrivateCommands=1 RecoveryThreads=40 # The DSN bellow references it remote equivalent in case the local TimesTen instance # fails. This DSN will be used by the BI Server in cases where it detects that the # local TimesTen database is no longer available. [TT_AGGR_STORE_REMOTE] TTC_SERVER = tt_exalytics_remote TTC_SERVER_DSN = TT_AGGR_STORE_LOCAL 5. Instantiate Oracle TimesTen Database. The TimesTen In-Memory Database is instantiated when the first connection is made to the database. cd /u01/app/oracle/product/timesten/tt1122/bin./ttisql connect dsn=tt_aggr_store_local; create user exalytics identified by password; grant create session to exalytics; grant create table to exalytics; grant select on SYS.OBJ$ to exalytics; Use the following command to test the connection to the local TimesTen In-Memory Database. There are no tables yet configured to return. 11 CONNECT DSN=TT_AGGR_STORE_LOCAL;uid=exalytics ; tables; EXIT; The Connectivity Test to the remote TimesTen In-Memory Database must be postponed until it has been installed and configured. Oracle WebLogic Server 1. Create boot.properties for Admin Server to start without prompting you for the administrator username and password in /u01/app/oracle/product/fmw/user_projects/domains/bifoundation_domain/servers/adminse rver/security. password= wls Admin User Password username= wls Admin User 2. Enable SDP Support for JDBC. a. Ensure that you have specified the virtual IP addresses used for InfiniBand network during BI Domain creation. The console automatically generates the complete JDBC URL as shown in the following example: For mds-owsm multi data source, we have 2 generic data sources: mds-owsm-rac0 URL: OL=TCP)(HOST= dm01db01- ibvip.mycompany.com)(port=1522)))(connect_data=(service_name= dbm)(instance_name=dbm1))) mds-owsm-rac1 URL: OL=TCP)(HOST= dm01db02- ibvip.mycompany.com)(port=1522)))(connect_data=(service_name= dbm)(instance_name=dbm2))) b. In the JDBC URL, replace TCP protocol with SDP protocol. For example: mds-owsm-rac0 URL: OL=SDP)(HOST= dm01db01- ibvip.mycompany.com)(port=1522)))(connect_data=(service_name= dbm)(instance_name=dbm1))) mds-owsm-rac1 URL: OL=SDP)(HOST= dm01db02- ibvip.mycompany.com)(port=1522)))(connect_data=(service_name= 12 dbm)(instance_name=dbm2))) c. Manually add the system property Djava.net.preferIpv4Stack=true to the startweblogic.sh script, which is located in the bin directory of DOMAIN_HOME as follows: Oracle Business Intelligence i. Locate the following line in the startweblogic.sh script:. ${DOMAIN_HOME}/bin/setDomainEnv.sh $* ii. Add the following property immediately after the above entry: JAVA_OPTIONS= ${JAVA_OPTIONS} Djava.net.preferIPv4Stack=true Doracle.net.SDP=true iii. Save the file and close it. iv. Restart the BI Domain. Configure ODBC Connection from Oracle Business Intelligence to Oracle TimesTen in /u01/app/oracle/product/fmw/instances/instance1/bifoundation/oraclebiapplication/coreapplication/s etup/odbc.ini to contain references to the TimesTen database. [ODBC Data Sources] AnalyticsWeb = Oracle BI Server Cluster = Oracle BI Server SSL_Sample = Oracle BI Server TT_AGGR_STORE_LOCAL = TimesTen Driver TT_AGGR_STORE_REMOTE = TimesTen Driver [TT_AGGR_STORE_LOCAL] Driver = /u01/app/oracle/product/fmw/../timesten/tt1122/lib/libttclient.so TTC_SERVER_DSN = TT_AGGR_STORE_LOCAL TTC_SERVER = tt_exalytics_local TTC_TIMEOUT = 0 [TT_AGGR_STORE_REMOTE] Driver = /u01/app/oracle/product/fmw/../timesten/tt1122/lib/libttclient.so TTC_SERVER_DSN = TT_AGGR_STORE_REMOTE TTC_SERVER = tt_exalytics_remote TTC_TIMEOUT = 0 Basic Exalytics Start-Stop Scripts first Node 13 These scripts are only examples and can be enhanced for better security and prerequisite checking as required. The goal of these examples is to provide the necessary commands and ordering for starting and stopping the processes on the Exalytics system. startexalytics.sh nohup /u01/app/oracle/product/fmw/user_projects/domains/bifoundation_domain/startweblogic.sh & sleep 60 nohup /u01/app/oracle/product/fmw/wlserver_10.3/server/bin/startnodemanager.sh & export ORACLE_INSTANCE=/u01/app/oracle/product/fmw/instances/instance1 /u01/app/oracle/product/fmw/instances/instance1/bin/opmnctl startall /u01/app/oracle/product/fmw/instances/instance1/bin/opmnctl status nohup /u01/app/oracle/product/fmw/user_projects/domains/bifoundation_domain/bin/startmanagedweb Logic.sh bi_server1 t3://exalytics1:7001 & /u01/app/oracle/product/timesten/tt1122/bin/ttdaemonadmin start stopexalytics.sh /u01/app/oracle/product/fmw/user_projects/domains/bifoundation_domain/bin/stopmanagedweb Logic.sh bi_server1 t3://exalytics1:7001 biadmin password export ORACLE_INSTANCE=/u01/app/oracle/product/fmw/instances/instance1 /u01/app/oracle/product/fmw/instances/instance1/bin/opmnctl stopall /u01/app/oracle/product/timesten/tt1122/bin/ttadmin ramunload TT_AGGR_STORE_LOCAL (See note below) /u01/app/oracle/product/timesten/tt1122/bin/ttdaemonadmin stop /u01/app/oracle/product/fmw/user_projects/domains/bifoundation_domain/bin/stopweblogic.sh biadmin password ps -ef grep NodeManager grep -v grep perl -lane 'print $F[1]' xargs kill -9 Note: It
Recommended
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x