ALEF Návod: Ako reinštalovať diskové pole FAS z Data ONTAP 7mode na Data ONTAP cluster mode

1. Zastavenie cluster failover a oboch nodov bežiaceho diskového poľa

FAS2240A> cf disable

FAS2240A> halt

FAS2240B> halt

2. Nastavenie prostredia

Keď sa zastavia oba nody a na oboch nodoch mam loader prompt, tak zadáme príkazy set-defaults pre vymazanie predchádzajúcich nastavení prostredia a pripravíme diskové pole pre cluster mode. Toto je zadať potrebné na oboch nodoch.

LOADER-A> set-defaults

LOADER-A> setenv bootarg.init.boot_clustered true

LOADER-A> setenv AUTOBOOT false

3. Odstránenie disk ownership pre použitie ADP pre root partitioning

Aby sa pri inštalácii Data ONTAP 8.3.1 cluster mode použil ADP na root partíciu, je potrebné odstrániť z diskov ownership. Toto som vykonal v „maintenance mode“. Nasledujúce kroky vykonáme na jednom node a potom aj na druhom node. Keď sa objaví boot menu, stlačíme Ctrl-C.

LOADER-A> boot_ontap

Loading X86_64/freebsd/image1/kernel:0x100000/9578776 0xa22918/4044416 Entry at 0x8016e880

Loading X86_64/freebsd/image1/platform.ko:0xdfe000/786856 0xf9bea0/724152 0xebe1c0/45064 0x104cb58/49752 0xec91c8/110791 0xee428f/80654 0xef7da0/172160 0x1058db0/195312 0xf21e20/16 0xf21e30/2448 0x10888a0/7344 0xf22800/0 0xf22800/344 0x108a550/1032 0xf22958/1952 0x108a958/5856 0xf230f8/1648 0x108c038/4944 0xf23768/240 0x108d388/720 0xf23860/448 0xf5e860/14942 0xf9bda2/253 0xf622c0/136824 0xf83938/99434

Starting program at 0x8016e880

NetApp Data ONTAP 8.2.4 Cluster-Mode

Could not get list of management ports for this platform!

Root mount waiting for: usbus0

…

Copyright (C) 1992-2015 NetApp.

All rights reserved.

md1.uzip: 39168 x 16384 blocks

md2.uzip: 16640 x 16384 blocks

*******************************

*                                                         *

* Press Ctrl-C for Boot Menu.                        *

*                                                         *

*******************************

^CBoot Menu will be available.

Please choose one of the following:

(1) Normal Boot.

(2) Boot without /etc/rc.

(3) Change password.

(4) Clean configuration and initialize all disks.

(5) Maintenance mode boot.

(6) Update flash from backup config.

(7) Install new software first.

(8) Reboot node.

Selection (1-8)? 5

You have selected the maintenance boot option:

The system has booted in maintenance mode allowing the following operations to be performed:

?                       disk

…

…

vsa                     xortest

Type "help " for more details.

In a High Availablity configuration, you MUST ensure that the partner node is (and remains) down, or that takeover is manually disabled on the partner node, because High Availability software is not started or fully enabled in Maintenance mode.

FAILURE TO DO SO CAN RESULT IN YOUR FILESYSTEMS BEING DESTROYED

NOTE: It is okay to use 'show/status' sub-commands such as 'disk show or aggr status' in Maintenance mode while the partner is up

Continue with boot? y

*> storage show disk

…

*> disk remove_ownership all

(Keby nefungoval prepínač all, tak je potrebné vymenovať disky jeden po druhom. Ja som popri tejto reinštalácii prehadzoval aj disky medzi diskovými poliami. V niektorých prípadoch mi fungoval parameter all, v iných som musel vymenovať disky.)

Po vykonaní tejto akcie na oboch nodoch zastavíme oba nody príkazom halt a z LOADER promptu sa znovu dostaneme do boot menu.

4. Nainštalovanie novej verzie Data ONTAP cez http server.

LOADER-A> boot_ontap

Loading X86_64/freebsd/image1/kernel:0x100000/9578776 0xa22918/4044416 Entry at 0x8016e880

Loading X86_64/freebsd/image1/platform.ko:0xdfe000/786856 0xf9bea0/724152 0xebe1c0/45064 0x104cb58/49752 0xec91c8/110791 0xee428f/80654 0xef7da0/172160 0x1058db0/195312 0xf21e20/16 0xf21e30/2448 0x10888a0/7344 0xf22800/0 0xf22800/344 0x108a550/1032 0xf22958/1952 0x108a958/5856 0xf230f8/1648 0x108c038/4944 0xf23768/240 0x108d388/720 0xf23860/448 0xf5e860/14942 0xf9bda2/253 0xf622c0/136824 0xf83938/99434

Starting program at 0x8016e880

NetApp Data ONTAP 8.2.4 Cluster-Mode

Could not get list of management ports for this platform!

Root mount waiting for: usbus0

…

Copyright (C) 1992-2015 NetApp.

All rights reserved.

md1.uzip: 39168 x 16384 blocks

md2.uzip: 16640 x 16384 blocks

*******************************

*                                                         *

* Press Ctrl-C for Boot Menu.                        *

*                                                         *

*******************************

^CBoot Menu will be available.

Please choose one of the following:

(1) Normal Boot.

(2) Boot without /etc/rc.

(3) Change password.

(4) Clean configuration and initialize all disks.

(5) Maintenance mode boot.

(6) Update flash from backup config.

(7) Install new software first.

(8) Reboot node.

Selection (1-8)? 7
 

Zvolíme inštaláciu nového software. Ako interface na prístup k image novej verzie Data  ONTAP som použil management port e0M. Tento port je samozrejme potrebné nastaviť. Použiť sa dá aj ktorýkoľvek dátový port. Keďže som sa chcel vyhnúť problémom na našej internej sieti, tak som bol priamo pripojený na management port nodu. Putty session som mal otvorenú na SP, kade som vykonával management a http cez tú istú fyzickú cestu ale na e0M port.  Uistite sa, že máte pripravený http server s príslušným image novej verzie Data ONTAP. IP adresa y.y.y.y bola u mňa adresa môjho notebooku, kde sa nachádzal image na IP adrese http://y.y.y.y:8080/831P1_q_image.tgz.

 

This procedure is not supported for Non-Disruptive Upgrade on an HA pair.

The software will be installed to the alternate image, from which the node is

not currently running. Do you want to continue? {y|n} y

In order to download the package, a temporary network interface needs to be configured.

Select the network port you want to use for the download (for example, 'e0a')  e0M

The node needs to reboot for this setting to take effect.  Reboot now? {y|n}

(selecting yes will return you automatically to this install wizard) y

Rebooting...

Here are the permissions on /mroot directory:

drwxr-xr-x  3 root  wheel  512 Oct 27 22:22 /mroot

/mroot directory doesn't exist or is it not writeable... aborting coverage dump

Terminated

.

Uptime: 1m23s

System rebooting...

▒

Phoenix SecureCore(tm) Server

Copyright 1985-2008 Phoenix Technologies Ltd.

All Rights Reserved

BIOS version: 8.3.0

Portions Copyright (c) 2008-2014 NetApp, Inc. All Rights Reserved

CPU = 1 Processors Detected, Cores per Processor = 2

Intel(R) Xeon(R) CPU           C3528  @ 1.73GHz

Testing RAM

512MB RAM tested

6144MB RAM installed

256 KB L2 Cache per Processor Core

4096K L3 Cache Detected

System BIOS shadowed

USB 2.0: MICRON eUSB DISK

BIOS is scanning PCI Option ROMs, this may take a few seconds...

 +++++++++++++++++++

Boot Loader version 4.3

Copyright (C) 2000-2003 Broadcom Corporation.

Portions Copyright (C) 2002-2014 NetApp, Inc. All Rights Reserved.

CPU Type: Intel(R) Xeon(R) CPU           C3528  @ 1.73GHz

LOADER-A> boot_ontap

Loading X86_64/freebsd/image1/kernel:0x100000/9578776 0xa22918/4044416 Entry at 0x8016e880

Loading X86_64/freebsd/image1/platform.ko:0xdfe000/786856 0xf9bea0/724152 0xebe1c0/45064 0x104cb58/49752 0xec91c8/110791 0xee428f/80654 0xef7da0/172160 0x1058db0/195312 0xf21e20/16 0xf21e30/2448 0x10888a0/7344 0xf22800/0 0xf22800/344 0x108a550/1032 0xf22958/1952 0x108a958/5856 0xf230f8/1648 0x108c038/4944 0xf23768/240 0x108d388/720 0xf23860/448 0xf5e860/14942 0xf9bda2/253 0xf622c0/136824 0xf83938/99434

Starting program at 0x8016e880

NetApp Data ONTAP 8.2.4 Cluster-Mode

Root mount waiting for: usbus0

…

Root mount waiting for: usbus0

Copyright (C) 1992-2015 NetApp.

All rights reserved.

md1.uzip: 39168 x 16384 blocks

md2.uzip: 16640 x 16384 blocks

*******************************

*                             *

* Press Ctrl-C for Boot Menu. *

*                             *

*******************************

Proceeding with Install

In order to download the package, a temporary network interface needs to be configured.

Enter the IP address for port e0M: x.x.x.x

Enter the netmask for port e0M: 255.255.255.0

Enter IP address of default gateway: x.x.x.1

What is the URL for the package? http://y.y.y.y:8080/831P1_q_image.tgz

What is the user name on "y.y.y.y", if any?

Checking network link... success.

Checking route to host "y.y.y.y"... success.

Attempting to reach y.y.y.y ... success.

Looking up URL "http://y.y.y.y:8080/831P1_q_image.tgz"... success.

Downloading package... success.

Checking package contents... success.

Extracting install script from package... success.

Invoking script (validation phase).

INSTALL running in check-only mode: the image will be validated only

Mode of operation is COPY

Current image is image1

Alternate image is image2

Package MD5 checksums pass

Available space on boot device is 1537 MB

Required space on boot device is 533 MB

Kernel binary matches install machine type

Terminated

Invoking script (install phase).  This may take up to 20 minutes.

Mode of operation is COPY

Current image is image1

Alternate image is image2

Package MD5 checksums pass

Available space on boot device is 1537 MB

Required space on boot device is 533 MB

Kernel binary matches install machine type

Getting ready to install image

Syncing device...

Extracting to /cfcard/x86_64/freebsd/image2...

Installed MD5 checksums pass

Installing diagnostic and firmware files

Firmware MD5 checksums pass

Terminated

Installation complete.

Do you want to set the newly installed software as the default to be used for subsequent reboots? {y|n} y

The node must be rebooted to start using the newly installed software. Do you want to reboot now? {y|n} y

Rebooting...

Here are the permissions on /mroot directory:

drwxr-xr-x  3 root  wheel  512 Oct 27 22:22 /mroot

/mroot directory doesn't exist or is it not writeable... aborting coverage dump

Waiting for PIDS: /usr/sbin/ypbind 767.

Waiting for PIDS: /usr/sbin/rpcbind 733.

Setting default boot image to image2... done.

Terminated

.

Uptime: 6m32s

System rebooting...

▒

Phoenix SecureCore(tm) Server

Copyright 1985-2008 Phoenix Technologies Ltd.

All Rights Reserved

BIOS version: 8.3.0

Portions Copyright (c) 2008-2014 NetApp, Inc. All Rights Reserved

CPU = 1 Processors Detected, Cores per Processor = 2

Intel(R) Xeon(R) CPU           C3528  @ 1.73GHz

Testing RAM

512MB RAM tested

6144MB RAM installed

256 KB L2 Cache per Processor Core

4096K L3 Cache Detected

System BIOS shadowed

USB 2.0: MICRON eUSB DISK

BIOS is scanning PCI Option ROMs, this may take a few seconds...

 +++++++++++++++++++

Boot Loader version 4.3

Copyright (C) 2000-2003 Broadcom Corporation.

Portions Copyright (C) 2002-2014 NetApp, Inc. All Rights Reserved.

CPU Type: Intel(R) Xeon(R) CPU           C3528  @ 1.73GHz

LOADER-A> setenv AUTOBOOT true

LOADER-A>

V tomto momente som ukončil prácu na node A a vykonal som nahranie nového image na node B. Je dôležité v tomto momente počkať, pretože ak by sa spustil „zeroing“ na disky na node A, tak by sa neaplikoval ADP (Advanced Disk Partitioning). Preto je lepšie vykonať natiahnutie novej verzie Data ONTAP aj na druhý node. Ďalšie kroky je potrebné vykonávať až po nahraní novej verzie Data ONTAP na oboch nodoch. 

5. Inicializácia diskov a vytvorenie ADP root partície

Po príprave oboch nodov dostaneme nový systém do boot menu a zvolíme menu 4, vytvoriť čistú konfiguráciu a inicializovať všetky disky. Túto činnosť spustíme na jednom a potom aj na druhom node, aby sa nody dohodli a vytvorili ADP root partície a rozdelili si ich.

LOADER-A> boot_ontap

Loading X86_64/freebsd/image2/kernel:0x200000/10088648 0xb9f0c8/4301024 Entry at 0x80271e20

Loading X86_64/freebsd/image2/platform.ko:0xfba000/1990365 0x11a0000/296352 0x11e85a0/273360

Starting program at 0x80271e20

NetApp Data ONTAP 8.3.1P1

Copyright (C) 1992-2015 NetApp.

All rights reserved.

*******************************

*                                                         *

* Press Ctrl-C for Boot Menu.                        *

*                                                         *

*******************************

Running the upgrade system

bootarg.upgrade="true"

bootarg.from.version="8.2.4"

bootarg.to.version="8.3.1P1"

^CBoot Menu will be available.

Please choose one of the following:

(1) Normal Boot.

(2) Boot without /etc/rc.

(3) Change password.

(4) Clean configuration and initialize all disks.

(5) Maintenance mode boot.

(6) Update flash from backup config.

(7) Install new software first.

(8) Reboot node.

Selection (1-8)? 4

ixgbe: e1a: ** JUMBOMBUF DEBUG ** switching to large buffers(9k -> 3k): (sz = 5120)!

ixgbe: e1b: ** JUMBOMBUF DEBUG ** switching to large buffers(9k -> 3k): (sz = 5120)!

WAFL CPLEDGER is enabled. Checklist = 0x7ff841ff

add host 127.0.10.1: gateway 127.0.20.1

Zero disks, reset config and install a new file system?: y

This will erase all the data on the disks, are you sure?: y

Rebooting to finish wipeconfig request.

Skipped backing up /var file system to CF.

Terminated

.

Uptime: 6m9s

System rebooting...

▒

Phoenix SecureCore(tm) Server

Copyright 1985-2008 Phoenix Technologies Ltd.

All Rights Reserved

BIOS version: 8.3.0

Portions Copyright (c) 2008-2014 NetApp, Inc. All Rights Reserved

CPU = 1 Processors Detected, Cores per Processor = 2

Intel(R) Xeon(R) CPU           C3528  @ 1.73GHz

Testing RAM

512MB RAM tested

6144MB RAM installed

256 KB L2 Cache per Processor Core

4096K L3 Cache Detected

System BIOS shadowed

USB 2.0: MICRON eUSB DISK

BIOS is scanning PCI Option ROMs, this may take a few seconds...

 +++++++++++++++++++

Boot Loader version 4.3

Copyright (C) 2000-2003 Broadcom Corporation.

Portions Copyright (C) 2002-2014 NetApp, Inc. All Rights Reserved.

CPU Type: Intel(R) Xeon(R) CPU           C3528  @ 1.73GHz

Starting AUTOBOOT press Ctrl-C to abort...

Loading X86_64/freebsd/image2/kernel:0x200000/10088648 0xb9f0c8/4301024 Entry at 0x80271e20

Loading X86_64/freebsd/image2/platform.ko:0xfba000/1990365 0x11a0000/296352 0x11e85a0/273360

Starting program at 0x80271e20

NetApp Data ONTAP 8.3.1P1

Copyright (C) 1992-2015 NetApp.

All rights reserved.

*******************************

*                                                         *

* Press Ctrl-C for Boot Menu.                        *

*                                                         *

*******************************

Wipe filer procedure requested.

original max threads=40, original heap size=41943040

bip_nitro Virtual Size Limit=80844390 Bytes

bip_nitro: user memory=748892160, actual max threads=42, actual heap size=44459622

ixgbe: e1a: ** JUMBOMBUF DEBUG ** switching to large buffers(9k -> 3k): (sz = 5120)!

ixgbe: e1b: ** JUMBOMBUF DEBUG ** switching to large buffers(9k -> 3k): (sz = 5120)!

WAFL CPLEDGER is enabled. Checklist = 0x7ff841ff

add host 127.0.10.1bootarg.bootmenu.selection is |4a|

: gateway 127.0.20.1

Cannot do remote rescan. Use 'run local disk show' on the console of ?? for it to scan the newly assigned disks

Dec 02 13:03:37 [localhost:diskown.RescanMessageFailed:warning]: Could not send rescan message to ??. Use the "disk show" command in nodeshell of ?? for it to scan the newly inserted disks.

.......................

 

Toto spustíme na oboch nodoch. Táto činnosť bude trvať dosť dlho, obvykle niekoľko hodín, v závislosti od počtu a typu diskov v systéme. Po jej ukončení je možné sa znovu pripojiť cez IP adresu servisného procesora. Na oboch nodoch bude spustený „cluster setup“ script a teda je možné vykonať konfiguráciu Data ONTAP v cluster mode.

6. Ešte jeden malý problém sa objavil

Na jednom z nodov sa mi po spustení „cluster setup“ skriptu objavila nasledujúca chybová správa:

::> cluster setup

Error: command failed: Exiting the cluster setup wizard. The root volume is

damaged. The cluster administrator's account (username "admin") password

is set to the system default. Contact support personnel for the root

volume recovery procedures. Run the "cluster setup" command after the

recovery procedures are complete.

 

Problém spočíval v tom, že tento node sa zasekol v recovery móde. Našťastie postačuje iba vymazať recovery príznak a systém sa naštartuje v poriadku. Aby sme to dosiahli, systém je najprv potrebné zastaviť.

::> halt -node local

Keď je systém v loader prompte, tak vymažeme recovery príznak a znovu ho naštarujeme

LOADER-A> unsetenv bootarg.init.boot_recovery

LOADER-A> boot_ontap

Po naštartovaní systém umožnil spustiť „cluster setup“ skript bez problémov.