Slow Imaging


Author
Message
5teve
5teve
New Member
New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)
Group: Awaiting Activation
Posts: 6, Visits: 15
HI Guys

Been using Macrium for a while now (about 8 years) had some issues with my 12 disk hardware raid 6 array which are now sorted. I am trying to image the raid array to a single 8tb disk which is connected to the same hardware raid card (HP P822 with 2gb cache and battery backup)

Normal network back up speed - server to HP microserver running freenas maxes out the network (1Gbps) so around 90-100MB/s transfer.
Crystal Mark Benchmark on the Raid array - 2.2GB/s sequential reads (with file size at 8gb so cache doesnt come into play)
8tb single disk (for backup) 200+MB/s write speed 

Macrium image back up speed - Raid to single disk - sitting on around 50MB/s so slower than network even.. certainly not saturating the write speed on the disk. Priority is high compression is off...

Something doesnt seem right?

Steve

jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)
Group: Forum Members
Posts: 7K, Visits: 51K
Does your 8TB disk use SMR (Shingled Magnetic Recording)?  Those disks incur a major performance penalty under prolonged uninterrupted write activity, which is precisely what you'd be doing in this scenario.

Edited 29 June 2020 5:16 AM by jphughan
5teve
5teve
New Member
New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)
Group: Awaiting Activation
Posts: 6, Visits: 15
jphughan - 29 June 2020 5:09 AM
Does your 8TB disk use SMR (Shingled Magnetic Recording)?  Those disks incur a major performance penalty under prolonged uninterrupted write activity, which is precisely what you'd be doing in this scenario.

HI

Thanks for the reply - no its a seagate (not my prefered but all that was available at short notice) Ironwolf so CMR and benchmarks very well..

Steve
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)
Group: Forum Members
Posts: 7K, Visits: 51K
If possible, can you attach it to a port not controlled by the RAID controller just to see if only using the controller for one side of this transaction rather than having it run high read and write I/O simultaneously makes a difference?
Edited 29 June 2020 5:22 AM by jphughan
5teve
5teve
New Member
New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)
Group: Awaiting Activation
Posts: 6, Visits: 15
jphughan - 29 June 2020 5:21 AM
If possible, can you attach it to a port not controlled by the RAID controller just to see if only using the controller for one side of this transaction rather than having it run high read and write I/O simultaneously makes a difference?

I Could do.. but being the card it is i know it has plenty of bandwidth.. as it can sustain 227 drives.. 

I will dig out a sata cable and see what it does.. even file transfers now dont seem very quick..
5teve
5teve
New Member
New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)
Group: Awaiting Activation
Posts: 6, Visits: 15
Bah my own bloody stupid fault!

Looks like when I created the array / logical drive (the P822 still classes a single drive as a raid 0 array) the default sector count was 32 not 63.. changed it to 63 re formatted with 128 cluster / 12k stripe size and i'm sitting happily at 250MB/s transfer rate and under 5 hours for 4.95tb!

Lesson to check my settings before looking to blame the software Smile

Im my defence the array / system has been running 24/7 for 5 years and i havent really touched it in that time.. so I have forgotten the info I used when I configured it Smile

Steve
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)
Group: Forum Members
Posts: 7K, Visits: 51K
Glad you've got it sorted, but when you say "default sector count", given that you're using the number 63, it sounds like you might mean starting sector?  Windows XP started used sector 63 as its starting sector, but that changed with Vista, and using a starting sector of 63 on disks that use 4K sectors results in poor performance, since starting on sector 63 causes the file system clusters not to fall along disk physical sector boundaries.  The performance hit is bad enough that manufacturers of Advanced Format disks offer "alignment" utilities so that people who initialized their AF disks on an XP system or cloned from a disk initialized that way could shift their sectors around in order to achieve optimal performance on their AF disk.  An 8TB disk would certainly be an AF disk, so if anything I'd have expected making that change in the OPPOSITE direction to solve the problem.

5teve
5teve
New Member
New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)
Group: Awaiting Activation
Posts: 6, Visits: 15
jphughan - 29 June 2020 1:48 PM
Glad you've got it sorted, but when you say "default sector count", given that you're using the number 63, it sounds like you might mean starting sector?  Windows XP started used sector 63 as its starting sector, but that changed with Vista, and using a starting sector of 63 on disks that use 4K sectors results in poor performance, since starting on sector 63 causes the file system clusters not to fall along disk physical sector boundaries.  The performance hit is bad enough that manufacturers of Advanced Format disks offer "alignment" utilities so that people who initialized their AF disks on an XP system or cloned from a disk initialized that way could shift their sectors around in order to achieve optimal performance on their AF disk.  An 8TB disk would certainly be an AF disk, so if anything I'd have expected making that change in the OPPOSITE direction to solve the problem.

HI JP
Its how the HP SSA refers to various items when creating an array (even a single disk is an array) so you have options to set the stripe size but also the sector count of the disk ( I think it refers to default sectors per track) - the default in SSA is for some reason 32 but the disk specs specifically state default is 63.. only info i can get now (as the array is up) is the drive details showing
Legacy Disk Geometry (C/H/S) 65535 / 255 / 63
Before it was Legacy Disk Geometry (C/H/S) 65535 / 255 / 32

See page 8 specs.. https://www.seagate.com/www-content/product-content/ironwolf/en-us/docs/100807039h.pdf

Either way the change made a massive difference and these 8tb disks are fast... compared my puny little 2tb HGST drives.. however 12 of them spin up some impressive speed for sequential writes.. certainly faster than 3x 8tb would.. bit more efficient with the raid 6 space too Smile

Hope thats makes a bit more sense.. as I mentioned its been a while so takes me a while to catch back up!

Steve
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)Macrium Evangelist (10K reputation)
Group: Forum Members
Posts: 7K, Visits: 51K
Ah ok, sectors per track is different from starting sector number of first partition.  I guess 63 just so happens to be a relevant value in both cases.  Or (more likely) Windows XP might have decided to use sector 63 as its start sector based on the common geometry of 63 sectors per track.  Still, I've never had to specify geometry details when setting up a RAID virtual disk.  It's typically just stripe size and then controller-level options like write caching.  But then again most of my RAID experience is on Dell servers and other NAS devices.  I'm not sure I've ever set up RAID on an HP box.  Very strange....

5teve
5teve
New Member
New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)New Member (7 reputation)
Group: Awaiting Activation
Posts: 6, Visits: 15
jphughan - 29 June 2020 2:14 PM
Ah ok, sectors per track is different from starting sector number of first partition.  I guess 63 just so happens to be a relevant value in both cases.  Or (more likely) Windows XP might have decided to use sector 63 as its start sector based on the common geometry of 63 sectors per track.  Still, I've never had to specify geometry details when setting up a RAID virtual disk.  It's typically just stripe size and then controller-level options like write caching.  But then again most of my RAID experience is on Dell servers and other NAS devices.  I'm not sure I've ever set up RAID on an HP box.  Very strange....

Haha that where it gets even more interesting.. its not a hp box!
As an experiment I was given a P822 with all the bells and whistle to use on a asrock rack c226 to see if it would work. Pain to get started but other than the bios not being accessible its fine.. supports firmware updates etc.. and is very very quick.. I'd love to see it go with some big SSD's  I say quick.. it was quick for its time.. i'm sure its just an OAP by todays standards but still works for me and a small business.

https://support.hpe.com/hpesc/public/docDisplay?docId=c03353708#N10058

Thanks again for your replies...

Steve
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search