[AIX] AIX Striping
* Striping Concept
The striping mechanism, also known as RAID 0, is a technology that was developed to achieve I/O performance gain. The basic concept of striping is that in the write phase, each data is chopped into small pieces(called striped unit, or chunk) and these chunks are written to separate physical volumes in paralled. In the read phase, these chunks are read from those separate physical volumes in parallel, and re-assembled into the actual data. From the high-availability point of the view, the striping function does not provide redundancy by any means.
-> redundancy를 제공하지 않음
Under figure, you have a volume group (stripevg) that is composed of three physical volumes having the same size (PV1, PV2, and PV3). A striped logical volume (stripelv) is created on this volume gorup, which spreads over the three physical volumes.
-> 각각의 PV에서 같은 사이즈를 할당함
* The logical partitions mapping scheme
In the striped logical volumes, all the logical partitions are mapped to physical partiitons on these physical volumes in a round-robin fashion.
This logical partitions to physical partitions mapping scheme is achieved by the inter-physical volume allocation set to maximum.
* The physical partitions are divided into many chunks(덩어리)
When your applications access the striped logical volumes, the storage area for each physical partition is not used contiguously(연속되게). These physical partitions are divided into chunks. The chunks size may be 4KB, 8KB, 16KB, 32KB, 64KB, or 128KB. This size is determined at the creation of the striped logical volumes, and cannot be changed after.
-> chunk는 stripe size를 의미하며, 이것은 한번 설정하면 변경이 불가능하다.
The chunk size is also called stripe unit size or stripe length. The number of the physical volumes that accommodate the striped logical volume is also called stripe width.
Under figure, each physical partition is divided into chunks. Each of them is represented as X-Y, where X is the physical partition number, and Y is the chunk occurrence number in the physical partition.
* Write phase
In the write phase, a caller (like the journaled file system device driver) issues one write I/O request to the AIX LVM device driver (step 1 under figure). This request is chopped(잘라지다) into several chunks (step 2), and they are written to each physical disk in parallel (step 3).
Since these three writes (to the chunks 1-1, 2-1, and 3-1) are achieved on separate drives, these writes are executed in parallel. It improves the write performance compared to one drive. Subsequently, if all the writes to each physical volume return with no error (step 4), then the LVM device driver returns a success to the caller (step 5). Otherwise, it returns an error to the caller.
* Read phase
In the read phase, a caller (like the journal file system device driver) issues one read I/O request to the AIX LVM device driver (step 1 under figure). This request is split into several read calls for chunks (step 2), and these small reads are sent to each physical disk in parallel. Since these three reads (from the chunks 1-1, 2-1, and 3-1) are achieved on separate drives, these reads are executed in parallel (step 3). It improves the read performance compared to one physical drive.
Subsequently, if all the reads to each physical volume return with no error (step 4), the LVM device driver directly passes the memory pointers for all the chunks that compose the data into the user's memory area without re-assembly (step 5). If one of the read calls for chunks failed, then the AIX LVM device driver would return an error to the caller.
* Inter physical volume allocation policy
The inter-physical volume allocation policy is one of the attributes of the logical volumes. It stipulates(규정하다) how many physical volumes are used for the alloation of the physial partitions for the logical volume. Usually (meaning non-striped logical volume case) it is set to m (minimum) by default to minimize the physical volumes that are used for the allocation of the physical partitions of the logical volume. It is adequate(적당한), since if one of the physical volumes that accommodates this logical volume fails, then the entire logical volume will be lost.
-> 몇개의 PV를 lv에서 사용할지 규정. non-striped환경에서는 minimum이맞음
* Stripe를 설정하지 않고 inter-policy를 maximum으로 설정한 잘못된 case
At first look, it looks like a striped logical volume. But, there is a distinct(전혀다른, 별개의) difference between this logical volume and a striped logical volume since there is no chunk level I/O striping benefit. Also, this physical volume allocation does not provide any I/O performance gain, but also it provides vulnerability_취약성(any disk fails and the logical volume is lost).
-> non-striped환경에서 inter-policy를 maximum으로 하면, performance적인 혜택은 전혀 없고, 오히려 하나의 disk fail시전체 lv를 잃을 수 있으므로 위험하다.(poor man's stripe)
This configuration is not beneficial in most circumstances, and it is called a poor man's stripe(not an actual striped logical volume).
* Pre-defined allocation policy
In a striped logical volume, the inter-physical volume allocation policy is forcibly pre-defined as maximum to achieve the logical partitions mapping scheme. It cannot be changed after the initial creation. If you try to force this value, then you would see the error message described in "Prohibited options of the mklv command with striping".
-> striped환경에서 inter-policy는 maximum이 default이며, 변경을 시도해도 error발생
(chlv: The -r, -e, -s, -d options cannot be used with striped logical volumes.)
* The reorganization relocation flag
The reorganization relocation flag is also one of the attributes of the logical volumes. It stipulates(규정하다) whether the logical volume can be relocated or not. For non-striped logical volume case, it is set to y(relocatable) to allow the relocation of the logical volume during the reorganization process.
-> RELOCATABLE: no로 자동으로 설정된다. 따라서 reorganization process(reorgvg)는 striped환경에서는 사용할 수 없다.
-> Bad Block Relocation(BB Policy)은상관없이 사용할 수 있다.
For striped logical volumes, the Relocate parameter must be set to n (the default for striped logical volumes).
* Relocation flag for striped logical volumes
On the striped logical volumes, the reorganization relocation flag is forcibly pre-defined as no.
If you try to force its value to yes, you will see following error message.
This means that the striped logical volumes are never reorganized during the execution of the reorgvg command. If the striped logical volume was reorganized, then the physical partitions allocation policy might not fit the logical partitions mapping scheme, "The logical partitions mapping scheme". It would cause the loss of the performance benefit of the striped logical volumes.
* Reorganization and migration
Reorganization and migration of physical partitions are two different tasks and shouldn't be confused. The reorganization (executed by the reorgvg command) is a procedure(진행) that repacks the actual location of physical partitions in one physical volume to generate contiguous allocation of the physical partitions. The migration of physical partitions (executed by the migratepv command) is a procedure that moves the entire physical partition from one physical volume to another within a volume group.
This reorganization relocation flag does not prohibit(금지하다) the migration of the physical disk that accommodates the striped logical volume.
-> reorganize는 pv내의 pp의 actual location을 contiguous하게 repack하는 것이고, migration은전체 pp를 다른 pv로 이동하는 것이다.
* Creation and extension of the striped logical volumes
This section provides some information and guidelines about the creation and extension(확장) of the striped logical volumes.
- Specify(조건으로서 지정하다) a multiple of the strip width as logical partition number during the initial creation of the striped logical volume.
- You cannot change the stripe width after the initial creation
- Extension is made by multiple of the strip width
-> stripe width 변경 가능함!
* Specify a multiple of the strip width during initial creation
If you attempt to create a striped logical volume, you can only specify a number of the logical partition that is a multiple of the strip width.
For example, under figure, if you attempt to create a striped logical volume on three physical volumes, you can only specify multiple of three as the logical partition number. If you sepcified anything else than these values, then it will round up to the next multiple(배수) of the width. If you specified two as the logical partition number, then it is round up to three.
-> 세 개의 pv이므로 3개로 strip width를 지정할 수 있고, 확장을 고려하여 그 배수로 지정이 가능하다. 다른 숫자는 안됨
These logical partitions allocated must be equally located on every physical volume composing the striped logical volume. In our example, we have a volume group composed of two physical volumes(under figure). One is 4.5GB(named hdisk2) and the other 9.1GB(named hdisk3). we choose 4MB as the physical partition size for this volume group, then there are 537 PPs on disk2 and 1084 PPs on disk3.
In this example, if we create a striped logical volume in this volume group, then only 1074 logical partitions (537 physical partitions per each physical volume) can be allocated to this striped logical volume. Otherwise, the attempt to create the striped logical volumes will fail. The remaining space of the physical volume hdisk3 (shown as the hatched portion under the horizontal bold line) cannot be used in striped logical volume. You can still use this space for non-striped logical volumes, but this can affect the I/O performance of the striped logical volumes.
-> Stripe를 구성하고 특정 disk에 남는 space가있다면, 이는 사용을 못한다. 이 space를 통해 lv를 non-striped로 생성하면 기존의 stripe lv에 I/O Performance측면의 영향을 줄 수 있다.
* Cannot change the stripe width
You cannot add new physical volumes to an already created striped volume. To do this, you have to recreate it (you have to create a backup of it and restore it). Because the actual data is already striped on the physical volumes, if new physical volumes are added to this striped logical volume, all the data that reside on physical volumes have to be re-striped to the newly added physical volume physical partitions.
Note: You cannot add new physical volume to already-created striped logical volumes. In another words, you cannot change the stripe width of an existing striped logical volume.
-> Striped volume에 새로운 pv를 추가하는 것은 불가능하다. striped vg를 recreation해야만 한다.
* Extension is made by stripe width base
To extend the striped logical volumes, you have to specify a multiple of the stripe width as logical partition number. Otherwise the attempt to extend the striped logical volumes will fail(under figure). In this case, since the striped logical volume scatters(흩뿌리다) over three physical volumes, the stripe width is three. Therefore, you have to specify a multiple of three as the extension logical partition number.
-> 배수로 stripe width를 지정해 두어야 확장이 가능하다.
* Prohibited(금지된) options of the mklv command with striping
To create the striped logical volume, you must use the -S option of the mklv command. This optional flag specifies that the logical volume is designated(명시하다) as a striped logical volume. There is a fundamental difference between the mirroring and striping function, of course, other than the functionality(기능성, 상관성) itself. You can always mirror an existing non-mirrored logical volume (including a striped logical volume from AIX Version 4.3.3), and also remove the mirror from the mirrored logical volumes. But, you cannot convert a non-striped logical volume to a striped logical volume, or vice versa(거꾸로, 반대로). The only way to create the striped logical volumes is to explicitly specify the -S flag on the mklv command line or use the corresponding SMIT panel.
* The exclusive optional flags of the -S flag for the mklv command are:
As a result of these exclusive optional flags, you do not have the ability to precisely(정확하게) control the physical partitions allocation of the logical volumes.
* Prohibited options of the extendlv command with striping
Some limitations also apply to the extendlv command, used to extend the logical volume size. If you attempt to extend the striped logical volumes, the following two command options are prohibited.
-m mapfile option
physical volume names
* Summary
As a conclusion for this section, it is recommanded that you keep in mind the following rules.
- Due to the lack of precise control for the physical partitions allocation, you may not place the striped logical volumes as you want. The best way to avoid this situation is to dedicate(봉납된) a volume group that accommodates the striped logical volumes. It also benefits the I/O performance.
-> I/O Performance benefits를 위해 VG를 Striped용도로 지정하여 사용하는 것이 좋다.
- If you create a striped logical volume, then you should choose disks that have same characteristics (especially size). Otherwise, you cannot use the entire surface of the physical volumes for the striped logical volumes.
-> 같은 용량의 pv를 사용하여 구성한다. 그래야 나중에 낭비하는 공간이 없다.
- If you cannot avoid using different sized physical volumes for creating the striped logical volumes, and, if you have to use the rest of the space of the larger size physical volume(s), you should minimize the I/O activity on that portion. Otherwise, that activity might affect the stiped logical I/O performance and you might loose the benefit of the striped logical volumes.
-> 불가피하게 다른 size의 pv를 사용했고, 남는 공간에 lv를 생성하고 싶다면 최대한 작은 I/O activity를 발생시키는 lv를 생성하라.
그렇지않으면 striped lv의 performance benefit를 잃게 된다.
* How to create the striped logical volumes
To create a striped logical volume, you can use the SMIT interface or the mklv command directly. In either case, there are three ways to specify the necessary information to create a striped logical volume.
- Specify all physical volume names directly
- Specify the maximum number of physical volumes allocated to the striped logical volume.
- Specify both of the options above.
The -u option of the mklv command is used to specify the maximum number of physical volumes used for the physical partition allocation of this striped logical volume. It is described in the command reference of the mklv command as follow:
-u UpperBound : Set the maximum number of physical volumes for new allocation. The value of the Upperbound variable should be between one and the total number of physical volumes. When using striped logical volumes or super strictness.
the upper bound indicates the maximum number of physcial volumes allowed for each mirror copy.
Note: When creating a super strict logical volume, you must specify physical volumes or use the -u flag.
You might get confused on which options are taken if both options are specified during the initial creation of a striped logical volume. However, the rule is simple. The -u option is just ignored if both options are spcified.
* How to extend the striped logical volumes
To extend the logical volumes (not only the striped one), you may have to increase the maximum logical partition number first (if the new logical volume size is greater than 512 logical partitions).
* The mirror and stripe function
AIX Version 4.3.3 introduces the new mirroring and striping function (also known as RAID 0+1 or RAID 10 technology). This function provide better data availability at the cost of extra disks.
One failing disk in a striped mirror copy does not bring down the entire logicla volume. The remaining (working) disks in the striped mirror copy continue to service striped units. Although the striped units on the failed disk are inaccessible, the mirrored copies of these units are available from another mirror. In this case, users are unaware that a disk is unavailable.
-> stripelv를 mirror하게되면 lslv속성에서 copy부분이 1에서 2로 변경된다. maximum number와 상관없이mirror는 실행되며, mirror가 수행되기 위한 여분의 disk만 존재하면 된다. 이제부터는 lp size를 increase하여도 mirror로 동작하게 된다.
upper bound value is not related to the number of mirror copies in case you are using RAID 10 configuration.
To support the mirroring and striping function, a new partition allocation policy (super strict) is introduced in AIX Version 4.3.3. This policy prohibits partitions from one mirror from sharing a disk with a second or third mirror. But, why is the super strict allocation policy needed for the mirror and stripe fucntion?
* Super strict allocation policy
AIX Version 4.3.3 introduces this new allocation policy type, super strict. This option is specified to the mklv command as the s flag for the -s option. Here are the possible flags for the strict option:
-s Strict : Determines the strict allocation policy. Copies of a logical volume can be allocated to share or not to share the same physical volume. The strict parameter is represented by one of the following:
y : Sets a strict allocation, so copies for a logical partition cannot share the same physical volume. This is the default for allocation policy.
n : Does not set a strict allocation policy. So copies for a logical partition cannot share the same physical volume.
s : Sets a super strict allocation policy, so that the partitions allocated for one mirror copy cannot share a physical volume with the partitions from another mirror copy.
-> Super strict allocation policy를 통해 lv의 copy가 copy되는 여러 disk들을 share해서 사용할 수 없게 해준다.
아래 그림을 보면 hdisk1은hdisk2로만, hdisk4는hdisk5로만 copy가된 것을 확인할 수 있다.
* Initial creation with mirror and super strict allocation policy
If you sepcified the super strict allocation policy in the previous example, you will have the following allocation of the physical partitions(under figure). You can force the super strict allocation policy with the -S s option on the command line.
This allocation is almost the same as the one shown in "Create the poor man's stripe first, then mirror", except for the allocated physical partition number. This allocation is not only desirable(바람직한), but also necessary for a correct management of a mirror and stripe logical volume.
The only way to guarantee the allocation of the physical partition represented in upper figure is by adopting(채택하다) the super strict allocation policy for the mirrored and striped logical volumes. In fact, this is the default value for the newly created striped logical volume, and it forces the super strict style allocation. Using this facility(설비), each mirror (shown with a different pattern of rectangles under figure) does not share the physical volumes with other mirrors. It enables you to add or remove mirror(s) from the striped logical volumes.
* How to manage a mirrored and striped logical volume
This section shows real examples of managing mirrored and striped logical volumes.
1) Example configurations
2) Real example of the creation
To create a mirrored and striped logical volume, you can choose one of the following ways:
- Create the striped logical volume, then mirror and synchronize it.
Create the striped logical volume (not mirrored yet) first, then create the mirror of this logical volume later. In this case, you have to instruct to synchronize each copy explicitly.
- Specify to create the mirrors during the initial creation.
Specify to create the mirrors of the striped logical volume upon the initial creation time.
* Create the striped logical volume, then mirror and synchronize it
1. Create the striped logical volume.
To create a striped logical volume, you have to explicitly specify the following information.
a. Where you want to create the striped logical volumes
You have to choose one of the followings.
1) The physical volume names.
You have to specify the physical volume names explicitly.
2) The maximum physical volume number.
You have to specify this number explicitly using the -u option of the mklv command or in the SMIT dialog.
3) Both of the above.
b. The stripe unit size
You have to specify this number explicitly using the -S option of the mklv command or in the SMIT dialog.
2. Create the mirror of the striped logical volume.
To create the striped logical volume, you have to explicitly specify the following information:
< From AIX 5L >
* Striped column support for logical volumes
Redundant Array of independent Disks (RAID) is a term used to describe the technique of improving data availability and data I/O rates through the use of arrays of disks and various data-striping methodologies. RAID algorithms can be implemented as part of the operating system's file system software, or as part of a disk device driver.
AIX LVM supports the following RAID options:
- RAID 0 Striping
- RAID 1 Mirroring
- RAID 10 or 0+1 Mirroring and striping
AIX 5L Version 5.3 further enhances the LVM RAID 0 and the LVM RAID 10 implementation and provides striped columns support for logical volumes. This new feature allows to extend a striped logial volume even if one of the physical volumes in the disk array became full.
-> AIX 5L(3)부터는 striped columns를 통해 하나의 pv가 full되더라도 lv의 size를 extend할 수 있다.
In previous AIX releases you could enlarge the size of a striped logical volume with the extendlv command as long as enough physical partitions were available within the group of disks which define the RAID disk array.
Rebuilding the entire LVwas the only way to expand a striped logical volume beyond the hard limits imposed(부과된) by the disk capacities. This work-around(예비수단, 차선택) required to backup and delete the striped LV and then to recreate the LV with a larger stripe width followed by a restore operation of the LV data.
To overcome the disadvantages of this time-consuming procedure, AIX 5L Version 5.3 introduces the concept of striped columns for LVs.
Prior to Version 5.3 the stripe width of a striped LV was determined at the time of LV creation by either of the following two methods:
- Direct specification of all physical volume names
- Specification of the maximum number of physical volumes allocated to the striped logical volume
The maximum number of PVs allocated to the striped LV is determined by the upper bound value specified with the -u flag of the mklv command. If you use the -u flag and also directly specify the PV names, the later takes precedence(우위, 선행, 우선권) and the -u flag will be ignored. It was not permitted to configure a striped logical volume with an upper bound larger than the stripe width.
-> 이전에는 upper bound가 stripe width보다 클 수 없었음.
In AIX 5L Version 5.3 the upper bound can be a multiple of the stripe width. One set of disks, as determined by the stripe width, can be considered as one striped column. Note that the upper bound value is not related to the number of mirror copies in case you are using RAID 10 configuration. Under illustrates the new concept of striped columns for a logical volume with stripe width 3 and an upper bound of 6.
-> 4개의 pv가 있는 vg에서 2개의 pv를 사용하여 striped lv를만들었을 경우 300개의 lp추가가 필요한데 용량이 부족하다면, upper bound를 4로 조정한뒤에 2개의 여분 pv를 추가로 사용하여 lv size를increase할 수 있다. 이 경우 기존 2개의 디스크의 여유공간과 함께 추가된 disk가stripe로 동작하게 된다. 처음 생성시 specific하게 디스크를 지정할 수 도 있고 random으로 생성할 수도 있다.
If you use the extendlv command to extend a striped logical volume beyond the physical limits of the first striped column, an entire new set of disks will be used to fulfill the allocation request for additional logical partitions. If you further expand the LV more striped columns may get added as required and as long as you stay within the upper bound limit . The chlv -u command allows you to increase the upper bound limit to provide additional headroom for striped LV expansion. The -u flag of the enhanced extendlv command can be used to raise the upper bound and extend the LV all in one operation.
Beginning with AIX 5L Version 5.3 the given upper bound value can be larger than the stipe width. If the striped LV has to be extended to the next column a full new stripe width of disks will be used; for that reason, the upper bound value must be specified in multiples of the stipe width . The following list describes the new -C command flag and the modified -u flag in more detail.
-C Stripe_width Sets the stripe width of the logical volume. If the Stripe_width variable value is not entered it is assumed to be the UpperBound variable value or the total number of disks specified on the command line.
-u UpperBound Sets the maximum number of physical volumes for new allocation. The value of the UpperBound variable should be between one and the total number of physical volumes. When using super strictness, the upper bound indicates the maximum number of physical volumes allowed for each mirror copy. When using striped logical volumes, the UpperBound value must be a multiple of the Stripe_width value . If upper bound is not specified, it is assumed to be Stripe_width for striped logical volumes.
Logical volume NAME [leetest]
* VOLUME GROUP name testvg3
* Number of LOGICAL PARTITIONS [100] #
PHYSICAL VOLUME names [hdisk2 hdisk3 hdisk4 hdisk7] +
Logical volume TYPE [jfs2] +
POSITION on physical volume outer_middle +
RANGE of physical volumes minimum +
MAXIMUM NUMBER of PHYSICAL VOLUMES [32] #
to use for allocation
Number of COPIES of each logical 1 +
partition
Mirror Write Consistency? active +
Allocate each logical partition copy yes +
on a SEPARATE physical volume?
RELOCATE the logical volume during reorganization? yes +
Logical volume LABEL []
MAXIMUM NUMBER of LOGICAL PARTITIONS [512] #
Enable BAD BLOCK relocation? yes +
SCHEDULING POLICY for writing/reading parallel +
logical partition copies
Enable WRITE VERIFY? no +
File containing ALLOCATION MAP []
Stripe Size? [128K] +
Serialize IO? no
-------------------------------------------------------------------------------------
LOGICAL VOLUME: leetest VOLUME GROUP: testvg3
LV IDENTIFIER: 00ce30ff00004c00000001355ff21604.8 PERMISSION: read/write
VG STATE: active/complete LV STATE: closed/syncd
TYPE: jfs2 WRITE VERIFY: off
MAX LPs: 512 PP SIZE: 16 megabyte(s)
COPIES: 1 SCHED POLICY: striped
LPs: 100 PPs: 100
STALE PPs: 0 BB POLICY: relocatable
INTER-POLICY: maximum RELOCATABLE: no
INTRA-POLICY: middle UPPER BOUND: 4
MOUNT POINT: N/A LABEL: None
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes (superstrict)
Serialize IO ?: NO
STRIPE WIDTH: 4
STRIPE SIZE: 128K
----------------------------------------------------------------------------------
* Logical volume NAME leetest
Logical volume TYPE [jfs2] +
POSITION on physical volume outer_middle +
RANGE of physical volumes maximum +
MAXIMUM NUMBER of PHYSICAL VOLUMES [8] #
to use for allocation
Allocate each logical partition copy superstrict +
on a SEPARATE physical volume?
RELOCATE the logical volume during reorganization? no +
Logical volume LABEL [None]
MAXIMUM NUMBER of LOGICAL PARTITIONS [512] #
SCHEDULING POLICY for writing/reading parallel +
logical partition copies
PERMISSIONS read/write +
Enable BAD BLOCK relocation? yes +
Enable WRITE VERIFY? no +
Mirror Write Consistency? active +
Serialize IO? no
댓글
댓글 쓰기