I’ve recently bumped into a couple painful limits of Data ONTAP. The first is the widely-known 16TB limit on aggregate size – with each day, it’s increasingly stunning to me that they haven’t increased this yet: 16TB is big, but it just isn’t that big any more. Just as painful is the much less discussed limits on A-SIS volume size – for the 2020 we just purchased, it’s 0.5TB, which caught us by surprise. Our reseller – who forgot to mention the A-SIS limit to us before we purchased the filer – claims that this limit will be at least doubled in ONTAP 7.3; 1TB is still lame.
We purchased the 2020 as a backup-to-disk device with two shelves of 750GB drives; when those 750GB drives shrink to the 621GB that’s actually usable from each to create an aggregate, we’re left in an uncomfortable position. We can create a 16TB aggregate but awkwardly have only a few drives left over that would either go unused or could be made into a really small aggregate. Or, we could guess what the best strategy to split the shelves up into two mid-sized aggregates would be – and hope we don’t paint ourselves into a corner later with this strategy. (The 2020 has a maximum of two external shelves – we’d have to head swap to a 2050 or better to add more.)
Since this is a backup-to-disk machine, A-SIS seemed a great fit for it, and we happily paid the license. Only now, we’re finding out that the largest volumes (or qtrees) we have – where we’d see the most benefit from A-SIS – can’t be deduplicated. I guess we should just consider this another case of the storage overhead tax that NetApp extracts – your 750GB drives become 621GB drives, your large volumes can’t be deduplicated, etc. I really hope that there’s a reasonable technical reason for the A-SIS limitation, and that it isn’t just to prop up NetApp’s margins through tough times by forcing their customers to buy more drives.
Unsolicited advice to NetApp: Quit suing Sun, stop spending time benchmarking EMC gear (as if anyone takes NetApp’s results on EMC seriously), get focused on fixing the problems in your own products and actually help your customers out for a change.
Andy,
One small piece of good news–the latest version of Data ONTAP for the FAS 2000 series, 7.2.4L1, adds the ability to have up to 4 external shelves on the FAS2020. So, you may have more headroom for growth than you think….
Scott Lowe
5 Feb 08 at 1:21 pm
That is good news about the ability to have more than two shelves on the 2020 in 7.2.4L1 – thanks for passing that along, Scott. (Incidentally, I’m a regular reader and big fan of your blog – thanks for all the effort you’ve put into that.)
Andy
Andy
5 Feb 08 at 1:48 pm
Yes, we have the similar, we were not informed of limits too. We purchase more shelves and eventually were forced to purchase new controllers. We can no longer revert to compitetors because of the initial investments.
With NetApp, you buy cheap initially and they’ll do the killing with unbelievable prices on your future purchases.
And since these are higly specialized equipment, there’s no way to compare your investment with that of the other companies.
lighter
5 Feb 08 at 4:41 pm
Andy,
Just wanted to shed some light on the A-SIS (now called Dedupe on FAS) volume limits. NetApp is keenly aware that this is a pain point for customers and working feverishly to try and raise the limits. And no its not a way to prop up NetApp’s profits (although maybe not a bad idea
) – there actually is a technical reason for it – in order to run volume maintenance tools (think WAFL check and WAFL iron) a certain amount of memory is needed. The more memory, the larger deduped volume can be supported. As you move through the FAS2000 line to the 3000 and 6000′s, you’ll see that larger and larger dedupe volumes are supported. Again, sorry you were surprised by this, but remember this feature has been released all of 10 months and good things are in store for the future.
Larry
Larry Freeman
5 Feb 08 at 4:43 pm
I am a customer who have both EMC and NetApp for a long time.
The original article written is very funny, biased. I think.
16TB being the limit for a single filesystem is really huge.
I am not sure what business application requires more that 16TB. If it was big filesystem how to backup and restore. What will happen to backup windows? No backup technology can backup at wire speed today to tapes.
First the box is 2020. It is the smallest what vendor can provide and you are expecting it to do great things. It is like getting a pussy cat from the market and telling it to behave like a lion.
ASIS dedupe works great for me on a 3000 series.
I may sound favouring NetApp. But the reality is the original article has no meaning.
Infact the EMC symmetrix systems we use have M1 and M2 disks for every practical disk I see.
Moreover the system does some kind of Normalizing. So 750GB seen as 620GB is pretty normal for all vendor. In somecases with other vendor I have seen 500GB as 200GB.
And as a customer who cares if NetApp benchmarks/Sue EMC or SUN or anyone. I don’t care.
For me, a system that helps the business is what it matters. So far NetApp systems has just done that. My Application team love the NetApp box.
Infact the last time I rebooted my NetApp was 1 year ago. It just runs.
Summary
=======
NetApp – Pretty Flexible, Robust system
KPC
30 Jun 08 at 10:44 am
Hi KPC,
Glad to hear that NetApp is a good fit for you and your company – that’s great, by all means, stick with what works for you! I’m also glad you found the piece funny – but I’m only laughing on the outside.
It sounds like our storage environment has different demands than yours. I work for a relatively small, but growing, research non-profit. When last I measured it, our data use was growing at a compound rate of about 8% each month; in other words, we double our storage use every nine months or so. (As we’re in the midst of a P2V project where direct-attached storage is moving to our NetApps, we’re actually growing faster than that now, but that’s a temporary bump.) We already have multi-terabyte volumes – so, you do the math… the 16TB aggregate limit is a real problem for sites like us.
It’s also worth noting that a 16TB aggregate is not a 16TB file system available to a server. 750GB SATA drives become Rightsize 621 GB drives. Then, for RAID-DP, subtract two disks out of each RAID group. Next, there’s the 10% WAFL overhead. And don’t forget to translate from marketing GB to real GB (or GB to GiB, if you will). So that maximum-size 26-disk aggregate made up of 750GB drives winds up as 11.4TB. And – of course – don’t forget your snap reserves after that.
As you mention, backups could be a challenge for large volumes; here’s how we solve it: The 2020 in question was purchased as a SnapVault secondary. Backups go from our primary 3040s to it, and then go via NDMP to tape for off-site/DR purposes. The secondary tier gives us the extended backup window we need to get the data to tape and meet our DR requirements. (I actually think this is a pretty common setup in this day and age.)
Of course, I’m not naive enough to think we can grow by adding drive shelves indefinitely (just added another one last Friday…). My personal opinion is that we’ll ultimately move to an HSM system, especially since much of the storage is used for instrument data (mass spec, microscopy, etc.) that is often difficult for researchers to categorize immediately as to its value. The thought is to let the HSM algorithms find the appropriate tier for the data automatically.
A couple other points:
– If you re-read the blog post, you’ll see that my main complaint with the 2020 was the lack of transparency with its limitations – and I know that I’m not alone in this department. (Heck, I just got a private email in the past week from a disappointed new 2020 owner.) Yeah, it’s low-end. But working for a non-profit, without the sometimes cushy budgets of corporate IT, we’ll buy what we can afford and push it hard; our funding agencies expect no less. Getting blindsided by limitations that we didn’t find out about during the sales process hurts. Should we have done more due diligence? Maybe, but the fact that we’re not alone in being unhappy with the 2020 suggests there’s a problem on the vendor side too.
– I think you’d do well to pay attention to the Sun/NetApp lawsuit. We use both NetApp and ZFS here, and I’m concerned that whichever side loses, it’s going to have an impact on our operations. I also think the lawsuit may have important implications for the storage industry as a whole.
Andy
30 Jun 08 at 12:54 pm
I understand your concern.
But as we say devil is in the detail. Really true.
Be it any vendor or anyone….a sales person always claim everything can fly.
We here do a proof of concept before making a decision.
In fact last year we did a PoC with other vendor NAS. They said it can achieve 300,000 iops.
End up our application can max get 40,000 iops. In the end it was a math that was about to do us in. There were multiple datamovers and they wanted us to use 8 filesystems across 8 datamovers to achieve that performance. Interesting confusing. End up it was a marketing hoax.
I think your FAS2020 is too low end for your environment. Looks like the reseller or person who size the system did not do it well. Your production being 3000 your snapvault should atleast be a min 3000. You are using storage for the first time looks like and this is pretty normal. If you try all the vendors in next 10 years, I tell you that you will feel NetApp was a better choice.
You can choose either raid-dp or raid-4. If you ask me a backup system can have raid4. So you gain lot of space. And snapreserves are the penalty you pay for quick restores. NetApp Snapshots are better than other online backup products.
My suggession will be that you directly ask NetApp to size your system and plan for you. Resellers are not as smart as we think. With low-budget a vendor can directly help if you have a long term plan with them.
And for your concern on Sun and NetApp, Nothing will happen. These are two big companies and small patent things will not impact you and me. Even Sun has to buy NetApp they will have to spend over 10B$. Sun for most part of Storage has hardly had any success. They got Storagetek but did not leverage it well. On the other hand NetApp is doing extremely well. I think these two companies will settle it off the court. You see even the SSD disk provider STEC is sued by Seagate. This is pure play of commercial politics.
KPC
30 Jun 08 at 6:43 pm
Hi KPC,
Thanks for commenting again – I got a good chuckle out of your “first” storage comment. My first storage array was a Digital StorageWorks box strung with a shared SCSI bus between two Sun servers in a HA cluster. Ah, the (not so) good old days.
You have a good point about doing a PoC, but in light of that, your comment about needing a 3000-series head for a SnapVault secondary of 3000-series primaries is an interesting assertion. We actually wound up moving from the 2020 to the 2050 purely because of the aggregate size limit (really, it does matter!), and the 2050 has worked admirably as a SnapVault secondary, A-SIS limitations aside.
I also disagree about RAID-4 being appropriate for a backup tier. Given modern drive sizes and error rates – particularly with SATA drives – I think single-parity RAID is an idea whose time has come and gone. But, then again, I realize that there are business situations where you need to do a cost-benefit analysis of things like single- versus double-parity RAID and the risks of data loss.
I may have been working with storage since the pre-fibre channel days, but I’ve only been working with NetApp for a couple of years. It’s good to hear that they’re keeping a long-term customer like yourself happy. However, with more of an outsider’s perspective, I see a company with a stagnant core product line and a next-gen version of ONTAP that’s perpetually just over the horizon – and their last quarter financials weren’t so hot, either.
Finally, I hope you’re right that the end-user doesn’t bear the brunt of the Sun-NetApp lawsuit. We’ll see soon enough, I suppose.
Andy
1 Jul 08 at 7:43 am
hi andy
i feel the same ! we have a new 2020 with a shelve (14x 1TB drives) at the end you just can use 2.72 TB – big joke
just my two cents
thx
diemsi
4 Jul 08 at 8:09 am
We purchased a FAS2020 earlier this year. Our reseller “forgot” to mention the 0.5TB volume limit for ASIS also. However, this hasn’t been much of a burden on us yet, although as we continue to expand our imaging systems it may be.
We mostly use the filer to support our little VI3 infrastructure with a few RDMs for some SQL/Imaging servers. Hopefully by the time we outgrow the 0.5TB limit Data OnTap 7.3 will be General Distribution release.
Other than this “oversight” and the outrageous prices I’m a big fan of NetApp.
I did find that it’s much easier to shop for NetApp gear towards the end of NetApp’s fiscal year/quarter. They seem to actually negotiate on price then.
Jim S
6 Aug 08 at 12:30 pm
If Netapp does not get A-SIS/Dedup sorted then check out Riverbeds new Atlas storage dedup devices when they come out of alpha/beta at the end of the year. Project named Corsica they are supposed to give up to 4x greater dedup than Netapp block level dedup due to the Riverbed byte level technology. We use both Riverbed AND Netapp and have to work around the multiple limits of dedup every day. It is very frustrating having a higher limit for dedup on one filer and then not being able to snapmirror to a slightly lower spec’d head due to a different LOWER limit volume dedup setting!
I know the technology is new (ish) but some consistency across the board would be helpful when configuring/architecting storage solutions. so far we have 3TB for 3040, 1TB for 3020 and 0.5TB for 2020….. ??
Riverbed Atlas will be CIFS only dedup on release but is slated for NFS (VMware Datastores) soon after.
Good luck
/kiwi
2 Oct 08 at 3:48 am
It’s probably not recommended to have a 16TB aggregate as with that number of drives, even with RAID-DP the likelyhood of failure of the whole lot is higher.
James White
17 Feb 09 at 5:00 am
I’m way late on this one, but anyway:
Andy & Jim: Re the A-SIS limit, with all due respect, if you bought un-aware of this limit then you didn’t do your due diligence. Yes, your reseller /definately/ should have mentioned it, but beyond that the limits were/are not exactly hidden – we bought a FAS2050 in June 08 and I remember the A-SIS limits being mentioned in multiple, prominent places (spec sheets, docs explaining A-SIS, etc.) while doing my research. At least 7.3.1 is out now so the limits have been raised somewhat!
diemsi – 14 x 1 TB disks for 2.72 TB usable? I think you’re doing something wrong there. Even if you follow “best practice” to the letter (RAID-DP, 2 hotspares) that should be more like 7TB available.
My 2 cents overall: We have a 2050 and I /love/ it, although admittedly I don’t have a lot to benchmark against it. It’s fast and expandable (for a company our size), A-SIS is achieving over 60% (!) average space saving across our volumes which equates to a lot of extra “free” storage, and NetApp snapshots are a huge time and effort saver.
Matt
23 Feb 09 at 1:53 am
Matt (or anyone) what sort of dedup throughput rates are you seeing in MB/sec on your FAS2050 (I guess it makes sense to run one dedup process per controller in most cases, or is anyone boldly running 8?)
Thanks
Jim
15 Jul 09 at 2:26 pm
Here we are in 2011, the FAS2020, the price point box Netapp was selling last year, can’t be upgraded to 64-bit ONTAP, thus we still have the limitation of 1 TB A-SIS.
I suggest for anyone considering Netapp as their new storage vendor, really look into the product. Don’t trust the reseller or Netapp, you have to do your own work. 2 months in to our Netapp fiasco I realized I should have Googled “limitations of the Netapp FAS2020″. I found about this after the reseller told me it was a limitation of the OS that will never change, because the box can’t support the 64-bit OS. I never saw a single document indicating this. I read of limitations, but none indicated the FAS2020 wouldn’t support ONTAP 8 (64-BIT). We wouldn’t have purchased this piece of crap knowing this. And the best part is, our Netapp rep said they could do a convenient buy-back option on the 2020 for one of the newer models. The funny thing is this device is 1 year old and they wanted to slap a 75% depreciation on it. They would offer us 25% off the new array if we swapped it.
It’s funny too, cause for several weeks our Netapp rep kept asking why we weren’t upgrading to ONTAP 8, when I told him the 2020A doesn’t support it, he stopped asking about it. So sales doesn’t even know about this. This is the low-end model, so they don’t care about it, just get it out the door.
I like the Netapp product and I do see value in it, however any potential ROI on this product is negated by the training required to administer the product and the disk space you lose to Netapp “features”.
I guess as long as you have money to burn and don’t buy their lowest option, Netapp is a great product.
If you are looking at Netapp, research everything before you even let them know you are looking.
And I will say this, Tier 1 support blows, you might as well just use Google. Unless your system is down, they are useless.
If you read Dave Hitz’ book, you would realize that from the very beginning Netapp was all about sales. It is apparent the companies philosophy was Sell it, no matter how limited the product is.
I suggest reading “How to Castrate a Bull” to get the real Netapp story.
Goid
24 Jan 12 at 10:57 am