Lab #1: Virtual Machine Setup - ProTech Training

15d ago
1.41 MB
27 Pages
Last View : 8d ago
Last Download : n/a
Upload by : Gia Hauser

Lab #1: Virtual Machine Setup Objective: This 2-hour lab guides you through the steps to download and configure the Cloudera Virtual Machine (VM) on your laptop. We will use this VM as the foundation of all lab exercises and demos. It is strongly recommended that all students complete this lab before the first day of class. Contact me via email at if you need any help with setting up the VM. Overview: The following high-level steps are involved in this lab: - Download Install VirtualBox Download Cloudera VM Configure the VM settings (memory/network) in VirtualBox Start Cloudera VM Install VirtualBox Guest Additions Verify Hadoop daemons are running If you’re running Windows, install PuTTY Note: Although I will only be supporting the Virtual Box environment in class, you may also run the Cloudera Virtual Machine under KVM, VMware or Amazon EC2 using Whirr. If you can’t use Virtual Box on your laptop, or prefer not to, check out the following Cloudera website with instructions for the alternate configurations: The most likely reasons for not being able to run the VirtualBox VM as outlined in this document are: - If your primary OS is 32-bit (the Cloudera VM is 64-bit and may not run in a 32-bit primary OS) Or if your laptop has less than 4 GB of RAM Or if your laptop hardware does not support VT-x/AMD-V virtualization (you can still try running the Cloudera VM if your laptop does not support VT-x/AMD-V, but it may not work) If any of these cases are true, consider launching a 64-bit Amazon EC2 medium instance (with 3.75 GB of RAM) for a few days and install CDH3U4 on it via a manual install or using Whirr. This should cost under 4 a day. As a final alternative, students who cannot run the VM will be paired with other volunteer students who have the VM successfully running.

Lab: Virtual Machine Setup, page 2 CDH3U4 Information: CDH Package used in VM: CDH3 Update 4 (released: May 9, 2012) A quote from “CDH consists of 100% open source Apache Hadoop plus nine other open source projects from the Hadoop ecosystem. CDH is thoroughly tested and certified to integrate with the widest range of operating systems, hardware and databases.” This Virtual Machine does not contain Cloudera Manager Free Edition or Enterprise Edition. CM Free Edition is a good choice for automated deployment, host-level monitoring and service management (start/stop) for clusters under 50 nodes. For a detailed comparison between Free and Enterprise edition, visit: Note: We will not be using the CDH4 package as it contains unstable code from Hadoop 2.0 branch with next-gen MapReduce and HDFS HA Federation. We will, however, discuss these architectural changes at the end of the course. This branch will become production ready at the end of 2012 or early 2013. Here is a list of the contents and versions for CDH3 Update 4: CDH3 Project Package Version Tarball Version Patch Level hadoop-0.20.2 923.256 hadoop-0.20.2-cdh3u4.tar.gz 923.256 flume-0.9.4 25.43 flume-0.9.4-cdh3u4.tar.gz 25.43 flume-ng-1.1.0 flume-ng-1.1.0-cdh3u4.tar.gz - hbase-0.90.6 84.29 hbase-0.90.6-cdh3u4.tar.gz 84.29 Hive hive-0.7.1 42.43 hive-0.7.1-cdh3u4.tar.gz 42.43 Hue hue- 114.35 hue-1.2.0-cdh3u4.tar.gz 114.35 mahout-0.5 9.5 mahout-0.5-cdh3u4.tar.gz 9.5 oozie-2.3.2 27.19 oozie-2.3.2-cdh3u4.tar.gz 27.19 pig-0.8.1 28.32 pig-0.8.1-cdh3u4.tar.gz 28.32 Sqoop sqoop-1.3.0 5.76 sqoop-1.3.0-cdh3u4.tar.gz 5.76 Whirr whirr-0.5.0 4.12 whirr-0.5.0-cdh3u4.tar.gz 4.12 zookeeper-3.3.5 19.1 zookeeper-3.3.4-cdh3u4.tar.gz 19.1 Hadoop 0.20 Flume Flume NG HBase Mahout Oozie Pig ZooKeeper

Lab: Virtual Machine Setup, page 3 For more detailed versioning and packaging information, visit: Version and Packaging Information Installing Oracle VirtualBox: VirtualBox is a powerful OS virtualization product that is freely available as open source software under GNU GPL version 2. VirtualBox runs on Windows, Linux and Macintosh hosts and supports a large number of guest operating systems including Windows (NT 4.0, 2000, XP, Server 2003, Vista, Windows 7), DOS/Windows 3.x, Linux (2.4 and 2.6), Solaris and OpenSolaris, OS/2, and OpenBSD. Here are some screenshots for VirtualBox: Download VirtualBox 4.1.16 for your host from here (approximately 100 MB): If you’re running Windows 7, download: VirtualBox-4.1.16-78094-Win.exe If you’re running OS X, download: VirtualBox-4.1.16-78094-OSX.dmg Install VirtualBox after downloading it. The installation is pretty straightforward and mostly involves clicking next a bunch of times. On the custom setup screen, leave all the default settings. On Windows 7, you may get 4 or 5 pop ups during the install from Oracle that you should just click “Install” to get past. For detailed installation instructions, you may refer to the official PDF: al.pdf If you’re very new to virtualization, you may want to check out this slightly outdated, but excellent introduction to VirtualBox at Lifehacker: to-creating-virtual-machines-with-virtualbox

Lab: Virtual Machine Setup, page 4 After the installation, VirtualBox should start with the following GUI: Read this if you’re running Windows (skip down if you’re running a Mac): Before proceeding to downloading and configuring the Cloudera VM, you should now enable VT-x or AMD-V virtualization in your computer’s BIOS. In the 1990s, virtualization was achieved by complex software techniques which overcame the processor’s lack of virtualization support. In the mid-2000s, both Intel and AMD have added hardware support to their processors, making virtualization software simpler and faster. Your laptop most likely supports hardware virtualization, but it probably needs to be turned on in the BIOS. Verify that it is turned on before proceeding. Sometimes this feature is called “Vanderpool” for Intel chips. If you’re running Windows, you can use this free software to check whether your laptop supports Hardware Virtualization:

Lab: Virtual Machine Setup, page 5 Read this if you’re running Mac OS X: Intel VT-x technology is enabled on Intel-based Macs. Make sure your Intel-based Mac has all available EFI firmware updates installed. See this Apple KB article for more information: Download Cloudera Virtual Machine: Now that you have VirtualBox up and running, it’s time to download set up the Cloudera VM. Download the Cloudera VM for VirtualBox from the following link. You will have to scroll down to the “CDH3 Packages and Downloads” section and then click on “Download for VirtualBox” from the table: File name to download (approximately 900 MB): cloudera-demo-vm-cdh3u4virtualbox.tar.gz Next, extract the gzip file and untar it. If you need help extracting the file on Windows, it is recommended to use 7-zip: On Mac OS X, you can use the gunzip command in terminal to extract the file or use a GUI tool like The Unarchiver: The final resulting file will be called (size: 2.5 GB): cloudera-demo-vm.vmdk Remember the location of the VMDK file as you will need it in the configuring section below. The next step is to configure VirtualBox for the Cloudera VM.

Lab: Virtual Machine Setup, page 6 Configure Cloudera Virtual Machine: Now let’s set up the VM in VirtualBox. This is a 64-bit image and requires at least 1.5GB of spare RAM (preferably 2GB) to run. So, it’s best to have a laptop with at least 4 GB of RAM, so that 2 GB can be allocated to the primary OS (Windows or Mac) and 2 GB to the guest OS (Linux Hadoop). If you’re running a 32-bit Windows OS, this VM might not work for you and may want to consider setting up a VM in Amazon as mentioned at the beginning of this document. The Cloudera VM is running the Xfce 4 desktop environment, which is a GUI for Linux, sort of like GNOME or KDE, but much more lightweight. Lots of effort was put into the VM to get all of the Hadoop daemons to run under severe memory pressure. We will use the following info to set up the VM: Name: Cloudera VM OS: Linux Version: Red Hat (64 bit) Memory: 2048 MB (note, this should be reduced to 1.5 GB (1536 MB of RAM) on low memory systems, but no less. Any less and some services will not start) Virtual Hard disk: Use Cloudera VM VMDK file

Lab: Virtual Machine Setup, page 7 Open the VirtualBox Manager GUI and click on New in the top right.

Lab: Virtual Machine Setup, page 8 Fill in the following VM details: Set the memory to 2 GB:

Lab: Virtual Machine Setup, page 9 Select Start-up Disk and choose “Use existing hard disk”. Choose the VMDK file from the extracted Cloudera VM download for the hard disk file. Finally, click “Create”.

Lab: Virtual Machine Setup, page 10 The VM currently should look like this, but don’t start the VM yet. We need to make a couple more configuration changes.

Lab: Virtual Machine Setup, page 11 Next, click on Settings in the Virtual Box Manager and change the following settings Under System, uncheck “Floppy” and verify that “Enable IO APIC” is checked: Leave the Processor and Acceleration tabs as default:

Lab: Virtual Machine Setup, page 12 Under Network, switch from NAT to Bridged Adapter (note, the Mac address on your laptop might be different, which is okay): Although the VM will be able to access the internet whether the above setting is NAT or Bridged, the difference is that with Bridged, the VM will get its own unique IP address from your Local Area Network’s router. This allows you to use PuTTY or Terminal to connect to the VM from your primary OS as you’ll see in the last section. With NAT, you have to input all commands for the VM via the VirtualBox GUI.

Lab: Virtual Machine Setup, page 13 Finally, under Shared Folders, set up a shared folder so that you can easily share files from your Host OS (Windows or Mac) to the Guest OS (Cloudera VM). You can name the folder anything you wish, but be sure to check “Auto-mount”: Click OK and close out of the VM Settings screen.

Lab: Virtual Machine Setup, page 14 Start Cloudera Virtual Machine: From the VirtualBox GUI, highlight the Cloudera VM and click Start. You will get a couple of popups about Auto capture keyboard and mouse pointer integration. You can click “Do not show this message again” and OK on these messages. Basically, they are saying that once your mouse is captured by the VM, you will have to click the Right Ctrl key in Windows (or another key, I think CMD in OS X) to release the mouse from the VM so you can continue to use it in the Primary OS.

Lab: Virtual Machine Setup, page 15 You can also ignore the 32-bit vs 16-bit color prompt for now.

Lab: Virtual Machine Setup, page 16 Finally, you should see the Cloudera VM running: Installing VirtualBox Guest Additions: The VirtualBox Guest Additions are a set of device drivers and system applications which may be installed inside the virtual machine. The Guest Additions assist with mouse pointer integration, shared folders, time synchronization and a shared clipboard. This is not critical to complete for our class, but it will make interacting with the VM a lot smoother.

Lab: Virtual Machine Setup, page 17 To install Guest Additions: Once the Cloudera VM is running, go to Devices - Install Guest Additions A CD should now appear on the Desktop with the Guest Additions software.

Lab: Virtual Machine Setup, page 18 Before installing Guest Additions, there are a few things we need to do. Open Add/Remove Software: Enter the password “cloudera” when promoted. You will next see a “Unable to retrieve software information” error. We will need to remove the Cloudera-cdh3 repository to get rid of this error. So, click on “Repository Manager”.

Lab: Virtual Machine Setup, page 19 In the Repository Manager, uncheck the Cloudera-cdh3 repository: Click close and soon you will see the “Package Manager”. Close out of the Package Manager for now. Open a Terminal Window inside the VM. Type su to become root and enter the ‘cloudera’ password. Then, type in yum update to update your guest OS.

Lab: Virtual Machine Setup, page 20 You will now see 3 or 4 packages that need to be updated. I saw the following 3 updates on May 28, 2012: Type in y to accept the updates and hit enter. When you see a message about importing a GPG key, type in y to accept and hit enter: Once the updates have successfully completed, you will see:

Lab: Virtual Machine Setup, page 21 Next, install GCC and kernel-devel (kernel development packages), which are needed by Guest Additions. You will have to choose ‘y’ for each to complete the install. To install GCC, type: yum install gcc Choose y when prompted. To install kernel-devel, type: yum install kernel-devel Choose y when prompted. Reboot your guest system in order to activate the updates. To reboot, type in: shutdown –r now After the system reboots, the Guest Additions CD should still be on the desktop. If it is not, go to Devices - Install Guest Additions and re-enable it. Type su and the ‘cloudera’ password to get to the root prompt:

Lab: Virtual Machine Setup, page 22 Navigate to the /media/VBOXADDITIONS 4.1.16 78094 directory with: cd /media/ VBOXADDITIONS 4.1.16 78094 Note, if you don’t see the VBOXADDITIONS folder under /media, try to re-enable the Guest Additions from Devices - Install Guest Additions. From the cmd prompt, as root, run: sh ./ During the install, you will get a prompt stating that the guest OS supports mouse pointer integration. Read through the message, choose “Do not show this message again” and click OK. This pop-up basically says that after a reboot, you no longer need to hit the right Ctrl to move the mouse pointer out of the VM. A successful installation of Guest Additions will look like this:

Lab: Virtual Machine Setup, page 23 Reboot your guest system in order to activate the Guest Additions. To reboot, type in: shutdown –r now After the VM reboots, you will notice that the mouse pointer seamlessly moves between your primary OS and the Guest VM. You will now also be able to copy paste between the two operating systems. The VM window, upon reboot, may also expand to take up more screen real estate. You can drag the edges of the window to shrink it back down if you’d like. Now, do a final check to ensure that all the default Hadoop daemons are running. Open up a cmd prompt within the VM and su to root with the ‘cloudera’ password. Then type this in to check which daemons are running: /usr/java/jdk1.6.0 21/bin/jps Note, you will not see the HBase or ZooKeeper daemons at this time. As long as you see all of the above daemons running, your Hadoop installation was successful. Installing Putty (Windows only): If you are on a Mac, skip this step. You can use the built in Terminal or iTerm2 (a more feature-rich replacement for Terminal) to connect to the VM, so you don’t need PuTTY.

Lab: Virtual Machine Setup, page 24 If you’re on Windows, I highly recommend installing PuTTY, a free telnet/SSH client. With PuTTY, you can connect to the Cloudera VM from a lightweight client in Windows and open multiple cmdline sessions to the VM. Download PuTTY from: sgtatham/putty/download.html Look for the file named putty.exe under “Windows on Intel x86”: There is no installation for PuTTY. You can just run it from the downloaded .exe file. After starting PuTTY, enter the IP address of the Cloudera VM into PuTTY. The connection type will remain SSH and the port will remain 22. To get the IP of the Cloudera VM, enter ifconfig into the Cloudera VM’s command prompt:

Lab: Virtual Machine Setup, page 25 And then paste that IP into PuTTY:

Lab: Virtual Machine Setup, page 26 Click “Open” to connect to the VM. You will have to click “Yes” to a message about the VM’s rsa2 key before a successful connection. The username and password for login are: Login as: cloudera cloudera@ your IP 's password: cloudera

Lab: Virtual Machine Setup, page 27 Final Notes: Congratulations! You have successfully set up the Hadoop lab environment for class. Please email me at to let me know that you’ve successfully completed this along with any feedback you have about my instructions.

The Cloudera VM is running the Xfce 4 desktop environment, which is a GUI for Linux, sort of like GNOME or KDE, but much more lightweight. Lots of effort was put into the VM to get all of the Hadoop daemons to run under severe memory pressure. We will use the following info to set up the VM: Name: OS: Linux Version: Red Hat (64 bit)

Related Documents:

Biology Lab Notebook Table of Contents: 1. General Lab Template 2. Lab Report Grading Rubric 3. Sample Lab Report 4. Graphing Lab 5. Personal Experiment 6. Enzymes Lab 7. The Importance of Water 8. Cell Membranes - How Do Small Materials Enter Cells? 9. Osmosis - Elodea Lab 10. Respiration - Yeast Lab 11. Cell Division - Egg Lab 12.

Contents Chapter 1 Lab Algorithms, Errors, and Testing 1 Chapter 2 Lab Java Fundamentals 9 Chapter 3 Lab Selection Control Structures 21 Chapter 4 Lab Loops and Files 31 Chapter 5 Lab Methods 41 Chapter 6 Lab Classes and Objects 51 Chapter 7 Lab GUI Applications 61 Chapter 8 Lab Arrays 67 Chapter 9 Lab More Classes and Objects 75 Chapter 10 Lab Text Processing and Wrapper Classes 87

decoration machine mortar machine paster machine plater machine wall machinery putzmeister plastering machine mortar spraying machine india ez renda automatic rendering machine price wall painting machine price machine manufacturers in china mail concrete mixer machines cement mixture machine wall finishing machine .

Each NETLAB remote PC or remote server runs inside of a virtual machine. VMware ESXi provides virtual CPU, virtual memory, virtual disk drives, virtual networking interface cards, and other virtual hardware for each virtual machine. ESXi also provides the concept of a virtual networking switch.

"Virtual PC Integration Components" software must be installed into each virtual machine. In a Windows host, the "Virtual PC Integration Components" software for a Windows virtual machine is located at C:\Program Files (x86)\Windows Virtual PC\Integration Components\ Multiple virtual machines can access the same target folder on the host.

The Java Virtual Machine Specification 5.6 Binding Native Method Implementations 401 5.7 Java Virtual Machine Exit 401 6 The Java Virtual Machine Instruction Set 403 6.1 Assumptions: The Meaning of "Must" 403 6.2 Reserved Opcodes 404 6.3 Virtual Machine Errors 404 6.4 Format of Instruction Descriptions 405 mnemonic 406 6.5 Instructions 408 .

(a) (b) Figure 3. The first version of Virtual Lab. (a) Wiring for the memory-seal logic. (b) Virtual Lab v1 was deployed on a Windows 8 tablet. One limitation of Virtual Lab v1 was the number and position of components was fixed. To address these problems, a second version (v2) of Virtual Lab was developed in 2014, which

Lab 5: Configuring VXLAN to provide network traffic isolation Lab 6: Introduction to OpenFlow Lab 7: SDN-routing within an SDN network Lab 8: Interconnection between legacy networks and SDN networks Lab 9: Configuring Virtual Private LAN Services (VPLS) with SDN networks Lab 10: