I’ve recently bumped into a couple painful limits of Data ONTAP. The first is the widely-known 16TB limit on aggregate size - with each day, it’s increasingly stunning to me that they haven’t increased this yet: 16TB is big, but it just isn’t that big any more. Just as painful is the much less discussed limits on A-SIS volume size - for the 2020 we just purchased, it’s 0.5TB, which caught us by surprise. Our reseller - who forgot to mention the A-SIS limit to us before we purchased the filer - claims that this limit will be at least doubled in ONTAP 7.3; 1TB is still lame.
We purchased the 2020 as a backup-to-disk device with two shelves of 750GB drives; when those 750GB drives shrink to the 621GB that’s actually usable from each to create an aggregate, we’re left in an uncomfortable position. We can create a 16TB aggregate but awkwardly have only a few drives left over that would either go unused or could be made into a really small aggregate. Or, we could guess what the best strategy to split the shelves up into two mid-sized aggregates would be - and hope we don’t paint ourselves into a corner later with this strategy. (The 2020 has a maximum of two external shelves - we’d have to head swap to a 2050 or better to add more.)
Since this is a backup-to-disk machine, A-SIS seemed a great fit for it, and we happily paid the license. Only now, we’re finding out that the largest volumes (or qtrees) we have - where we’d see the most benefit from A-SIS - can’t be deduplicated. I guess we should just consider this another case of the storage overhead tax that NetApp extracts - your 750GB drives become 621GB drives, your large volumes can’t be deduplicated, etc. I really hope that there’s a reasonable technical reason for the A-SIS limitation, and that it isn’t just to prop up NetApp’s margins through tough times by forcing their customers to buy more drives.
Unsolicited advice to NetApp: Quit suing Sun, stop spending time benchmarking EMC gear (as if anyone takes NetApp’s results on EMC seriously), get focused on fixing the problems in your own products and actually help your customers out for a change.
11 Comments
Andy,
One small piece of good news–the latest version of Data ONTAP for the FAS 2000 series, 7.2.4L1, adds the ability to have up to 4 external shelves on the FAS2020. So, you may have more headroom for growth than you think….
That is good news about the ability to have more than two shelves on the 2020 in 7.2.4L1 - thanks for passing that along, Scott. (Incidentally, I’m a regular reader and big fan of your blog - thanks for all the effort you’ve put into that.)
Andy
Yes, we have the similar, we were not informed of limits too. We purchase more shelves and eventually were forced to purchase new controllers. We can no longer revert to compitetors because of the initial investments.
With NetApp, you buy cheap initially and they’ll do the killing with unbelievable prices on your future purchases.
And since these are higly specialized equipment, there’s no way to compare your investment with that of the other companies.
Andy,
Just wanted to shed some light on the A-SIS (now called Dedupe on FAS) volume limits. NetApp is keenly aware that this is a pain point for customers and working feverishly to try and raise the limits. And no its not a way to prop up NetApp’s profits (although maybe not a bad idea :)) - there actually is a technical reason for it - in order to run volume maintenance tools (think WAFL check and WAFL iron) a certain amount of memory is needed. The more memory, the larger deduped volume can be supported. As you move through the FAS2000 line to the 3000 and 6000’s, you’ll see that larger and larger dedupe volumes are supported. Again, sorry you were surprised by this, but remember this feature has been released all of 10 months and good things are in store for the future.
Larry
I am a customer who have both EMC and NetApp for a long time.
The original article written is very funny, biased. I think.
16TB being the limit for a single filesystem is really huge.
I am not sure what business application requires more that 16TB. If it was big filesystem how to backup and restore. What will happen to backup windows? No backup technology can backup at wire speed today to tapes.
First the box is 2020. It is the smallest what vendor can provide and you are expecting it to do great things. It is like getting a pussy cat from the market and telling it to behave like a lion.
ASIS dedupe works great for me on a 3000 series.
I may sound favouring NetApp. But the reality is the original article has no meaning.
Infact the EMC symmetrix systems we use have M1 and M2 disks for every practical disk I see.
Moreover the system does some kind of Normalizing. So 750GB seen as 620GB is pretty normal for all vendor. In somecases with other vendor I have seen 500GB as 200GB.
And as a customer who cares if NetApp benchmarks/Sue EMC or SUN or anyone. I don’t care.
For me, a system that helps the business is what it matters. So far NetApp systems has just done that. My Application team love the NetApp box.
Infact the last time I rebooted my NetApp was 1 year ago. It just runs.
Summary
=======
NetApp - Pretty Flexible, Robust system
Hi KPC,
Glad to hear that NetApp is a good fit for you and your company - that’s great, by all means, stick with what works for you! I’m also glad you found the piece funny - but I’m only laughing on the outside.
It sounds like our storage environment has different demands than yours. I work for a relatively small, but growing, research non-profit. When last I measured it, our data use was growing at a compound rate of about 8% each month; in other words, we double our storage use every nine months or so. (As we’re in the midst of a P2V project where direct-attached storage is moving to our NetApps, we’re actually growing faster than that now, but that’s a temporary bump.) We already have multi-terabyte volumes - so, you do the math… the 16TB aggregate limit is a real problem for sites like us.
It’s also worth noting that a 16TB aggregate is not a 16TB file system available to a server. 750GB SATA drives become Rightsize 621 GB drives. Then, for RAID-DP, subtract two disks out of each RAID group. Next, there’s the 10% WAFL overhead. And don’t forget to translate from marketing GB to real GB (or GB to GiB, if you will). So that maximum-size 26-disk aggregate made up of 750GB drives winds up as 11.4TB. And - of course - don’t forget your snap reserves after that.
As you mention, backups could be a challenge for large volumes; here’s how we solve it: The 2020 in question was purchased as a SnapVault secondary. Backups go from our primary 3040s to it, and then go via NDMP to tape for off-site/DR purposes. The secondary tier gives us the extended backup window we need to get the data to tape and meet our DR requirements. (I actually think this is a pretty common setup in this day and age.)
Of course, I’m not naive enough to think we can grow by adding drive shelves indefinitely (just added another one last Friday…). My personal opinion is that we’ll ultimately move to an HSM system, especially since much of the storage is used for instrument data (mass spec, microscopy, etc.) that is often difficult for researchers to categorize immediately as to its value. The thought is to let the HSM algorithms find the appropriate tier for the data automatically.
A couple other points:
- If you re-read the blog post, you’ll see that my main complaint with the 2020 was the lack of transparency with its limitations - and I know that I’m not alone in this department. (Heck, I just got a private email in the past week from a disappointed new 2020 owner.) Yeah, it’s low-end. But working for a non-profit, without the sometimes cushy budgets of corporate IT, we’ll buy what we can afford and push it hard; our funding agencies expect no less. Getting blindsided by limitations that we didn’t find out about during the sales process hurts. Should we have done more due diligence? Maybe, but the fact that we’re not alone in being unhappy with the 2020 suggests there’s a problem on the vendor side too.
- I think you’d do well to pay attention to the Sun/NetApp lawsuit. We use both NetApp and ZFS here, and I’m concerned that whichever side loses, it’s going to have an impact on our operations. I also think the lawsuit may have important implications for the storage industry as a whole.
I understand your concern.
But as we say devil is in the detail. Really true.
Be it any vendor or anyone….a sales person always claim everything can fly.
We here do a proof of concept before making a decision.
In fact last year we did a PoC with other vendor NAS. They said it can achieve 300,000 iops.
End up our application can max get 40,000 iops. In the end it was a math that was about to do us in. There were multiple datamovers and they wanted us to use 8 filesystems across 8 datamovers to achieve that performance. Interesting confusing. End up it was a marketing hoax.
I think your FAS2020 is too low end for your environment. Looks like the reseller or person who size the system did not do it well. Your production being 3000 your snapvault should atleast be a min 3000. You are using storage for the first time looks like and this is pretty normal. If you try all the vendors in next 10 years, I tell you that you will feel NetApp was a better choice.
You can choose either raid-dp or raid-4. If you ask me a backup system can have raid4. So you gain lot of space. And snapreserves are the penalty you pay for quick restores. NetApp Snapshots are better than other online backup products.
My suggession will be that you directly ask NetApp to size your system and plan for you. Resellers are not as smart as we think. With low-budget a vendor can directly help if you have a long term plan with them.
And for your concern on Sun and NetApp, Nothing will happen. These are two big companies and small patent things will not impact you and me. Even Sun has to buy NetApp they will have to spend over 10B$. Sun for most part of Storage has hardly had any success. They got Storagetek but did not leverage it well. On the other hand NetApp is doing extremely well. I think these two companies will settle it off the court. You see even the SSD disk provider STEC is sued by Seagate. This is pure play of commercial politics.
Hi KPC,
Thanks for commenting again - I got a good chuckle out of your “first” storage comment. My first storage array was a Digital StorageWorks box strung with a shared SCSI bus between two Sun servers in a HA cluster. Ah, the (not so) good old days.
You have a good point about doing a PoC, but in light of that, your comment about needing a 3000-series head for a SnapVault secondary of 3000-series primaries is an interesting assertion. We actually wound up moving from the 2020 to the 2050 purely because of the aggregate size limit (really, it does matter!), and the 2050 has worked admirably as a SnapVault secondary, A-SIS limitations aside.
I also disagree about RAID-4 being appropriate for a backup tier. Given modern drive sizes and error rates - particularly with SATA drives - I think single-parity RAID is an idea whose time has come and gone. But, then again, I realize that there are business situations where you need to do a cost-benefit analysis of things like single- versus double-parity RAID and the risks of data loss.
I may have been working with storage since the pre-fibre channel days, but I’ve only been working with NetApp for a couple of years. It’s good to hear that they’re keeping a long-term customer like yourself happy. However, with more of an outsider’s perspective, I see a company with a stagnant core product line and a next-gen version of ONTAP that’s perpetually just over the horizon - and their last quarter financials weren’t so hot, either.
Finally, I hope you’re right that the end-user doesn’t bear the brunt of the Sun-NetApp lawsuit. We’ll see soon enough, I suppose.
hi andy
i feel the same ! we have a new 2020 with a shelve (14x 1TB drives) at the end you just can use 2.72 TB - big joke
just my two cents
thx
We purchased a FAS2020 earlier this year. Our reseller “forgot” to mention the 0.5TB volume limit for ASIS also. However, this hasn’t been much of a burden on us yet, although as we continue to expand our imaging systems it may be.
We mostly use the filer to support our little VI3 infrastructure with a few RDMs for some SQL/Imaging servers. Hopefully by the time we outgrow the 0.5TB limit Data OnTap 7.3 will be General Distribution release.
Other than this “oversight” and the outrageous prices I’m a big fan of NetApp.
I did find that it’s much easier to shop for NetApp gear towards the end of NetApp’s fiscal year/quarter. They seem to actually negotiate on price then.
If Netapp does not get A-SIS/Dedup sorted then check out Riverbeds new Atlas storage dedup devices when they come out of alpha/beta at the end of the year. Project named Corsica they are supposed to give up to 4x greater dedup than Netapp block level dedup due to the Riverbed byte level technology. We use both Riverbed AND Netapp and have to work around the multiple limits of dedup every day. It is very frustrating having a higher limit for dedup on one filer and then not being able to snapmirror to a slightly lower spec’d head due to a different LOWER limit volume dedup setting!
I know the technology is new (ish) but some consistency across the board would be helpful when configuring/architecting storage solutions. so far we have 3TB for 3040, 1TB for 3020 and 0.5TB for 2020….. ??
Riverbed Atlas will be CIFS only dedup on release but is slated for NFS (VMware Datastores) soon after.
Good luck
Post a Comment