HPE Serviceguard For Linux Enterprise Edition 12.30.00 .
HPE Serviceguard for Linux Enterpriseedition 12.30.00 Release NotesPart Number: P04491-002Published: July 2018
ContentsOverview. 6Supported platforms and Linux distributions.7Packaging information. 8Licensing information. 9Obtaining a permanent license. 9Renewing the permanent license.9Validating the permanent license. 10HPE Serviceguard for Linux Enterprise edition updates. 12Features introduced in this version. 12Features introduced in earlier versions.13Features introduced in A.12.20.00. 13Features introduced in A.12.10.00. 13Features introduced in A.12.00.51. 14Features introduced in A.12.00.50. 14Features introduced in A.12.00.40. 15Features introduced in A.12.00.30. 16Features introduced in A.12.00.22. 16Features introduced in A.12.00.21. 16Features introduced in A.12.00.20. 16Features introduced in A.12.00.00. 16Common features of Enterprise bundle. 18Metrocluster with Continuous Access EVA P6000 for Linux. 19Metrocluster with Continuous Access XP P9000 for Linux. 19Features not supported.20Metrocluster with 3PAR Remote Copy for Linux. 20Metrocluster with Continuous Access EVA P6000 for Linux. 20Metrocluster with Continuous Access XP P9000 for Linux. 21Metrocluster with EMC SRDF. 21Continentalclusters.21Compatibility matrix.21Installation.21Defects fixed . 22Known problems and limitations. 22Deprecated or obsolete features.22References.22Compatibility and installation requirements. 24Hardware requirements. 24Port requirements. 24Ports needed for Serviceguard. 242Contents
Ports needed for authentication. 24Ports needed by Serviceguard Manager.25System firewalls. 25Supported browsers. 27Software prerequisites for Serviceguard for Linux. 28Installing Serviceguard for Linux. 31Installing Serviceguard for Linux using cminstaller. 31Installing Serviceguard for Linux using cmeasyinstall.33Installing Serviceguard for Linux the traditional way.35Installing or upgrading Serviceguard for Linux using HPE Software Delivery Repository. 36Installing Serviceguard for Linux Update Release. 37Post installation.40Rolling software upgrade.41Requirements.41Limitations of rolling upgrades. 43Preparation. 44Rolling upgrade of a minor OS version of Linux. 45Rolling upgrade of major OS version of Linux. 46Major OS rolling upgrade of two node cluster running HPE Serviceguard for OracleData Guard.48Supported rolling upgrade paths for Serviceguard versions. 50Performing rolling upgrades of Serviceguard versions. 50Rolling upgrade from 11.20.X to 12.20.00 .51Rolling upgrade from 11.19.X to 12.20.00.56Performing offline rolling upgrade from 11.18.X to 12.20.00. 57Rolling upgrade from 12.00.X to 12.30.X. 58Rolling upgrade from A.11.19.X to A.12.20.X.65Offline rolling upgrade from A.11.18.X to A.12.20.X.65Upgrading Serviceguard for Linux packages. 66Removing Serviceguard for Linux. 67Troubleshooting.68Related information. 73Documentation feedback. 74Contents3
NoticesThe information contained herein is subject to change without notice. The only warranties for HewlettPackard Enterprise products and services are set forth in the express warranty statements accompanyingsuch products and services. Nothing herein should be construed as constituting an additional warranty.Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use,or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer SoftwareDocumentation, and Technical Data for Commercial Items are licensed to the U.S. Government undervendor's standard commercial license.Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett PackardEnterprise has no control over and is not responsible for information outside the Hewlett Packard Enterprisewebsite.4Notices
AcknowledgmentsIntel , Itanium , Pentium , Intel Inside , and the Intel Inside logo are trademarks of Intel Corporation in theUnited States and other countries.Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in theUnited States and/or other countries.Adobe and Acrobat are trademarks of Adobe Systems Incorporated.Java and Oracle are registered trademarks of Oracle and/or its affiliates.UNIX is a registered trademark of The Open Group.Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.Red Hat is a registered trademark of Red Hat, Inc. in the United States and other countries.Acknowledgments5
OverviewThis document provides information about HPE Serviceguard for Linux Enterprise edition 12.30.00.6Overview
Supported platforms and Linux distributionsServiceguard for Linux Enterprise edition 12.30.00 is available on the following Linux distributions: Red Hat Enterprise Linux 6 Red Hat Enterprise Linux 7 SUSE Linux Enterprise Server 11 SUSE Linux Enterprise Server 12NOTE: For more information about supported updates, supported hardware, storage, and other information,see the latest version of HPE Serviceguard for Linux Certification Matrix Enterprise Edition at pported platforms and Linux distributions7
Packaging informationServiceguard for Linux 12.30.00 is available on all three editions, namely, Serviceguard for Linux Base,Serviceguard for Linux Advanced, and Serviceguard for Linux Enterprise. Serviceguard for Linux Enterpriseedition 12.30.00 July 2018 contains rpms for the following components:8 Serviceguard License Serviceguard for Linux Serviceguard for Linux snmp Serviceguard for Linux WBEM providers Serviceguard for Linux analytics Serviceguard Quorum Server Serviceguard Manager Serviceguard Toolkit for Enterprise DB PPAS for Linux HPE Serviceguard Toolkit for Oracle Database for Linux HPE Serviceguard Toolkit for SAP Sybase ASE and Sybase Replication Server for Linux Serviceguard Extension for SAP Serviceguard XDC for Linux Metrocluster with Continuous Access XP P9000 for Linux Metrocluster with 3PAR Remote Copy for Linux Metrocluster with SADR for Linux Metrocluster with Continuous Access EVA P6000 for Linux Serviceguard Toolkit for KVM for Linux Metrocluster with EMC SRDF HPE Serviceguard Toolkit for Oracle Data Guard for Linux HPE Serviceguard Toolkit for Db2 for Linux HPE Serviceguard Solutions for Microsoft SQL Server for LinuxPackaging information
Licensing informationStarting Serviceguard for Linux 12.00.00 requires licenses on per-socket basis. When you installServiceguard for Linux Enterprise edition, an instant-on license valid for 90 days is installed. With thisinstant-on license, you can use the product even if you do not have a permanent license. You must get apermanent license before the grace period expires.When ordering the licenses, determine the number of active sockets on the server and order one license foreach active socket irrespective of number of cores. A virtualized server may select less than the totalamount of active sockets if Serviceguard is used within virtual machine which utilized less than the totalnumber of sockets. For information about the license terms and supported server models, see theQuickSpecs available at ml.IMPORTANT: If you plan to upgrade to new OS version, you can use the same license that you arecurrently using. For example, if you are upgrading from Red Hat Enterprise Linux 6 to Red HatEnterprise Linux 7, you can use the same license of Red Hat Enterprise Linux 6 on Red Hat EnterpriseLinux 7.Obtaining a permanent licenseBefore your 90 days instant-on license expires, you must obtain and validate the permanent license tocontinue to use applicable Serviceguard versions beyond the grace period.To obtain a permanent license:1. Go to https://myenterpriselicense.hpe.com.2. Log into HPE Passport. If you do not have an account, you can create one.3. Enter Entitlement Order Number and click Go. Here are the licenses listed that have been activated and license that havenot yet been activated message appears.4. Check the box that belongs to products you want to activate and click Next.5. Select “if you are activating for yourself” or “if you are activating on behalf of another customer”. If “onbehalf of another customer” is selected, you will enter the final user email address.6. Activation Completes. Save the files. One includes the license key and the other includes additionalproduct information.7. You will receive a license certificate in your email box. You must retain the email message because thisis the valid proof of purchase documentation you may need for future reference or support requests.IMPORTANT: Ensure that you save the file and make a note of its path. (See the example inValidating the permanent license on page 10.)Renewing the permanent licenseYou can renew the permanent license from the Serviceguard Manager.To renew the license from the CLI after you have obtained the permanent license, follow these steps torenew the license:Licensing information9
NOTE: You must renew or apply the license on each node.Procedure1. Ensure that SGCONF/AutoPass/LicFile.txt exists.NOTE: Hewlett Packard Enterprise recommends that you back up the SGCONF/AutoPass/LicFile.txt before renewing the license.2. Place the permanent license key in a file on the system.NOTE: Ensure the file that contains the license key must not have the commented line.3. Renew the license:#cmsetlicense -i absolute path of the license file On successful license renewal, it displays the following message:License is successfully installedFor more information, see cmsetlicense(1m) manpage.ExampleTo verify if the license is valid:a. Copy the license file:cp SGCONF/AutoPass/LicFile.txt SGCONF/AutoPass/LicFile.txt.savb. Run the cmsetlicense command:cmsetlicense -i /test/mySGlicenseThe mySGlicense file contains the license information.c. Run the cmgetlicense command:cmgetlicense -f lined. If the license is valid, keep the new license file SGCONF/AutoPass/LicFile.txt and delete theold license file SGCONF/AutoPass/LicFile.txt.sav.If the license is invalid, restore the original license:mv SGCONF/AutoPass/LicFile.txt.sav SGCONF/AutoPass/LicFile.txtValidating the permanent licenseIMPORTANT: Ensure that you validate the permanent license before the 90–day grace period ofinstant-on license expires.To validate the permanent license:1. Run the following command:10Validating the permanent license
#cmgetlicense -f line2. On successful renewal, the command displays the following output:#cmgetlicense -f linenode:node1 license type Enterprise license type Enterprisenode:node1 license type Enterprise license count 1node:node1 license type Enterprise license valid for days IndefiniteNOTE: If you install higher license than the one already installed on the system, the license on the node isupgraded automatically.For example, if you have Serviceguard for Advanced and install Serviceguard for Enterprise, thelicense on the node is upgraded to Enterprise and the cmgetlicense -f line command displaysthe following output:node:node1 license type Enterprise license type Enterprisenode:node1 license type Enterprise license count 1node:node1 license type Enterprise license valid for days Indefinite If you have multiple licenses of the same type installed on the system, the cmgetlicense -f linecommand displays the following output:node:node1 license type Enterprise license type Enterprisenode:node1 license type Enterprise license count 3node:node1 license type Enterprise license valid for days IndefiniteLicensing information11
HPE Serviceguard for Linux Enterprise edition updatesFeatures introduced in this versionServiceguard A.12.30.00 supports the following features: Serviceguard is safe against the Spectre V2 vulnerability. Starting with Serviceguard Version 12.30.00,Serviceguard complies with the OS vendor specified requirements to protect the system against SpectreV2 vulnerability. With this release, you can use a single command cminstaller to perform a fresh installation orupgrade HPE Serviceguard bundles. HPE Serviceguard bundle is available in three variants such asHPE Serviceguard for Linux Base, HPE Serviceguard for Linux Advanced and HPE Serviceguard forLinux Enterprise.For more information about cminstaller commands see the see the Upgrading Serviceguard forLinux using cminstaller tool. With this release, you can implement Oracle single instance, Oracle Data Guard, and both AOFI andAOAI workloads for Microsoft SQL Server on nonvolatile memory on nonvolatile memory (NVM) systemssuch as NVDIMM and NVMe memory systems. These memory systems are accessed as block devicesand bridge the performance and persistence gaps between the traditional DRAM and NAND flashsystems. You can store the database, transaction/redo logs, and the temporary files such as hot files,temporary table space, database cache files in the nonvolatile memory systems. Or you can choose tosave the combination of any of the data types on the traditional and the nonvolatile memory systems. Generic resource monitoring mechanism has been enhanced to support cluster generic resource. This isan easy to administer resource monitoring mechanism to monitor the critical resources of a cluster. It canbe used to monitor common resources of multiple packages at cluster level and depending upon thestatus of the resource, you can take appropriate actions at the package level. Use the cluster genericresource to integrate any third party, custom, or user-defined monitoring agents at the cluster level. You can implement Serviceguard Metrocluster on 3PAR Synchronous Long Distance (SLD) environment.In this configuration Serviceguard can seamlessly orchestrate the primary and secondary roles in a SLDenvironment and ensure high availability and disaster recovery between sites.Serviceguard Metrocluster provides RPO sensitive failover within the cluster. It also monitors thereplication status across all the three sites and provides user notification appropriately.12 You can upgrade a major version of Linux operating system and the Serviceguard software on a node ata time without causing downtime to your applications. For more information see, Rolling upgrade ofmajor OS version of Linux. Storage Agnostic Disaster Recovery (SADR) is a DR solution that is agnostic to the underlying arrayreplication and the array type. SADR enables automated failover and failback capability between primarydata center and secondary data center, which are connected in replication mode. Functionalities of theMetrocluster, such as role reversal, RPO/RTO monitoring, and automated recovery can be used with thissolution.HPE Serviceguard for Linux Enterprise edition updates
NOTE: Download and install SADR template RPM hosted at ion/SGLX-DR/A.12.30.00/null to use this feature. This RPM contains two files: SADR template file ReadME fileFeatures introduced in earlier versionsFeatures introduced in A.12.20.00Serviceguard A.12.20.00 supports the following features: Serviceguard provides a robust monitoring system to monitor all disks of a ASM mirrored disk groupconfigured in a Serviceguard package. Serviceguard discovers all the underlying disks of the ASMmirrored disk group and starts monitoring the disks to check their health. If a disk fails in a disk group,Serviceguard notifies the failure to the end user in the form of alerts and notifications. When the faileddisk is restored, Serviceguard makes it available to the disk group. HPE Serviceguard Solutions for Microsoft SQL Server for Linux provides High Availability and DisasterRecovery solutions for Microsoft SQL Server for Linux. With this toolkit, Serviceguard monitors the healthof the database and provides high availability and disaster recovery solutions for Availability Groups orAlways on Availability Groups (AOAI) deployment model.Features introduced in A.12.10.00Serviceguard A.12.10.00 supports the following features: To validate and test the preparedness of disaster recovery site without disrupting the operations ofproduction site, Serviceguard for Linux allows the selection of production workloads at site, node, andpackage levels for their recovery tests conducted at recovery site. It also prepares a detail report whichcaptures the details of various preparedness tests conducted for audit or compliance purposes. Serviceguard for Linux configured cluster nodes in VMware ESXi environment are compatible withVMware Site Recovery Manager (SRM) supported recovery plan executions. With this integration, itfacilitates automated disaster recovery with continued HA protection for applications independent of theirSRM site of operation. VMware’s Virtual Machine File System (VMFS) based virtual disks can be configured for ServiceguardMetrocluster with EMC SRDF for Linux based Metrocluster for disaster recovery. For easy viewing of multiple packages and their state which need to examined together to understandthe state of a solution level service delivery, Workload view is now available from single resource pane ofServiceguard Manager. You can create the following workload types: Oracle Single Instance DB Oracle Single Instance DB using ASMServiceguard Metrocluster with 3PAR Remote Copy for Linux supports SSH based connections only with3PAR arrays starting from Inform OS 3.3.1 (Supported from Serviceguard A.12.00.51 and later).Features introduced in earlier versions13
Features introduced in A.12.00.51 Serviceguard A.12.00.51 supports the cluster configuration for Virtual Machines deployed on VMwareESXi hosts enabled with Distributed Resource Scheduler(DRS). The DRS manages the allocation ofphysical resources to a set of virtual machines deployed in a cluster of hosts, each running VMware'sESXi hypervisor. Serviceguard supports Application Tuner Express (ATX). Metrocluster with 3PAR Remote Copy for Linux supports only SSH connection to the 3PAR arrays fromInform OS 3.3.1 onwards.Features introduced in A.12.00.50Oracle Data Guard Toolkit for LinuxWith Serviceguard 12.00.50 release, a new disaster recovery solution for Oracle workloads based on Oracledataguard replication is introduced. With this toolkit, Serviceguard manages the Oracle database and Oracledataguard replication. It performs RPO sensitive automatic role management (failover, switch over) tohandle failures, and also supports event notifications.For more information, see the HPE Serviceguard Toolkit for Oracle Data Guard on Linux User Guideavailable at trocluster with 3PAR Remote Copy for LinuxSupport for physical servers with disaster recovery solution based on 3PAR peer persistence.Metrocluster with EMC SRDF for LinuxMetrocluster with EMC SRDF Symmetrix Remote Data Facility is now supported on the following Linuxdistributions: Red Hat Enterprise Linux (RHEL) 7 server SUSE Linux Enterprise Server (SLES) 12Serviceguard Metrocluster for Linux with EMC SRDF B.01.00.00 is now bundled with Serviceguard for LinuxEnterprise Edition A.12.00.50 as Serviceguard Metrocluster for Linux with EMC SRDF B.12.00.50. All thefeatures that were supported with Metrocluster EMC SRDF B.01.00.00 are now available for MetroclusterEMC SRDF B.12.00.50 in addition to the following: Support for Red Hat Enterprise Linux (RHEL) 7 server Support for SUSE Linux Enterprise Server (SLES) 12Serviceguard Metrocluster for Linux with EMC SRDF B.01.00.00 is no longer available as a standaloneproduct for Serviceguard for Linux Base and Advanced editions.Following features that were supported with Metrocluster EMC SRDF B.01.00.00 are now bundled with12.00.50 release: Support for Synchronous and Asynchronous Replication modeMetrocluster with EMC SRDF for Linux includes support for synchronous replication mode. EMC SRDFsoftware offers synchronous and asynchronous data transfer mode between the storage systems.Synchronous data transfer offer the highest levels of data availability. With synchronous operations, bothprimary and secondary copies are identical and concurrent at all times. Synchronous replication isappropriate when data currency is critical while recovering a business application. In the asynchronous14Features introduced in A.12.00.51
mode, host writes are performed only on the primary storage array, and the host write is acknowledgedas soon as the data is written on the primary storage array. Supported for LVMThe LVM is supported with EMC SRDF disk group for Linux.For more information on supported versions of the volume managers, see the Serviceguard DisasterRecovery Products Compatibility and Feature Matrix(EMC SRDF disk group) available at http://www.hpe.com/info/linux-serviceguard-docs. Support for ContinentalclustersEMC SRDF replication is supported with Continentalclusters and can be used to achieve push buttonautomated recovery of EMC SRDF replication pairs. For more information, see the Continentalclustersfor Linux Version B.01.00.00 Release Notes available at http://www.hpe.com/info/linux-serviceguarddocs. Support for failover polices site preferred and site preferred manualFor more information on these failover polices site preferred and site preferred manual, see theMetrocluster SRDF for Linux B.01.00.00 Guide available at http://www.hpe.com/info/linuxserviceguard-docs. Support for persistent reservations Support for Device MapperFeatures introduced in A.12.00.40 Extended Distance Cluster (XDC) provides support for XDC based LVM Mirroring (RAID 1). Metrocluster with Continuous Access EVA P6000 for Linux provides automated failover of Serviceguardpackages on local and remote P6000/EVA series disk arrays. Support for VMware Virtual Machine File System (VMFS) on the following Metrocluster products: Metrocluster with 3PAR Remote Copy for Linux Metrocluster with Continuous Access EVA P6000 for Linux Metrocluster with Continuous Access XP P9000 for Linux Extended Distance Cluster (XDC)Metrocluster with 3PAR Remote Copy for Linux provides the following features: Support for Recovery Point Object (RPO). Support for Asynchronous Streaming Mode Support for peer persistence environment only if VMware Virtual Machine File System (VMFS)module is configured in the package configuration file. A disaster recovery solution based on 3PARpeer persistence with VMware guest nodes.Features introduced in A.12.00.4015
Features introduced in A.12.00.30Support for Smart Quorum. For more information on Smart Quorum, see Managing HPE Serviceguard A.12.00.30 for Linux.Support for Asymmetric node configuration in a disaster recovery solution. The following Metrolcusterproducts support asymmetric node configuration for sites: Metrocluster with 3PAR Remote Copy for Linux Metrocluster with Continuous Access EVA P6000 for Linux Metrocluster with Continuous Access XP P9000 for LinuxFeatures introduced in A.12.00.22There are no new features in this version of the release. The patch contains only defect fixes for SGeSAP.Features introduced in A.12.00.21There are no new features introduced in this version of the release.Features introduced in A.12.00.20 Site Aware Disaster Tolerant Architecture (SADTA) — SADTA is a framework, which enables you toeasily deploy disaster recovery solutions for complex multi-instance workloads. You can createServiceguard cluster to deploy complex workloads with SADTA using Serviceguard Manager. You canalso use this framework to build disaster recovery solution for SAP HANA scale out workloads. Metrocluster with 3PAR Remote Copy for Linux supports Failsafe mode of 3PAR array.Features introduced in A.12.00.00Metrocluster with 3PAR Remote Copy for Linux enables automatic failover of Serviceguard packagesbetween primary and recovery sites. In addition, Metrocluster with 3PAR Remote Copy for Linux providesthe following features: Support for Remote Copy volume group monitorIn a Metrocluster 3PAR environment, Remote Copy volume group (RCVG) )monitor provides thecapability to monitor the status of the Remote Copy volume group used in a package. It sendsno
Metrocluster with 3PAR Remote Copy for Linux Metrocluster with SADR for Linux . amount of active sockets if Serviceguard is used within virtual machine which utilized less than the total number of sockets. For information about the license terms and sup
HPE ProLiant DL180 Gen9 HPE ProLiant DL360 Gen9 HPE ProLiant DL380 Gen9 HPE ProLiant DL560 Gen9 HPE ProLiant ML30 Gen9 HPE ProLiant ML110 Gen9 HPE ProLiant ML150 Gen9 HPE ProLiant ML350 Gen9 Apollo 4200 Apollo 4500 Apollo 6000 . HP Smart Array P440/2GB Controller . HPE ProLiant DL20 Gen9 HPE ProLiant DL80 Gen9 HPE ProLiant DL120 Gen9
QuickSpecs HPE USB Keyboard/Mouse Kit Overview HPE ProLiant DL180 G6 Page 3 HPE ProLiant DL180 Gen9 HPE ProLiant DL20 Gen9 HPE ProLiant DL320 G6 HPE ProLiant DL320e Gen8 HPE ProLiant DL320e Gen8 v2 HPE ProLiant DL360 G7 HPE ProLiant DL360e Gen8 HPE ProLiant DL360p Gen8
medical conditions that could occur during a variety of physical activities n State Standards HPE 4 HPE 4 HPE 4 HPE 4 HPE 4 HPE 4 HPE 4 HPE 4 HPE 4 HPE 4 National Standards NPE 2, NPE 5 NPE 2, NPE 5 NPE 2, NPE 5 NPE 2 NPE 2 NPE 2 NPE 2 NPE 2 NPE 2 NPE 2
QuickSpecs HPE Ethernet 1Gb 4-port 331T Adapter Compatibility Page 2 Compatibility Servers ProLiant DL (rack-optimized): HPE ProLiant DL160 Gen8 HPE ProLiant DL165 G7 HPE ProLiant DL320e Gen8 HPE ProLiant DL360e Gen8 HPE ProLiant DL360p Gen8 HPE ProLiant DL380e Gen8 HPE ProLiant DL380p Gen8 HPE ProLiant DL385p
QuickSpecs HPE Ethernet 10Gb 2-port 560FLR-SFP Adapter . Compatibility . Page 3 . Servers . ProLiant DL (rack-optimized): HPE ProLiant DL120 Gen9 HPE ProLiant DL160 Gen9 HPE ProLiant DL180 Gen9 HPE ProLiant DL360 Gen9 HPE ProLiant DL380 Gen9 HPE ProLiant DL560 Gen9 HPE ProLiant
HPE ProLiant BL460c Gen9 HPE ProLiant BL660c Gen9 HPE ProLiant WS460c Gen9 HPE ProLiant BL460c Gen8 HPE ProLiant BL465c Gen8 HPE ProLiant BL660c Gen8 NOTE: This is a list of supported servers. Some may be discontinued. Compatibility - Supported Interconnect Modules HPE Virtual Connect FlexFabric-20/40 F8 Module for c-Class BladeSystem
HPE MSA 1040 SAN Storage HPE MSA 1050 SAN Storage HPE MSA 2040 SAN Storage HPE MSA 2050 SAN Storage Operating Systems Microsoft Windows Server Microsoft Windows Hyper-V Server VMware vSphere Red Hat Enterprise Linux (RHEL) SUSE Linux Enterprise Server (SLES) CentOS ClearOS NOTE: HPE Smart Ar
Annual Men’s Day Sunday, October 26, 2008 Order of Service Devotion Men’s Day Praise Team Responsive Reading Psalm 1 Congregational Singing This Little Light of Mine Selection Men’s Day Choir Altar Prayer Dance Selection Creations of God Announcements Bro. Robert Shields Pastoral Observations Message Rev. Jeffery A. Lang, Senior Pastor Southside Church & International Ministries Jackson .
