Okay, it looks like we've gotten to the top of the hour there may be some more people that joined a little bit late but that's fine let's go ahead and get started with today's webinar. So hello everyone and welcome. First of all thank you very much for taking time out of your busy schedule to step away with us for a little bit while we discuss databases and storage. So understanding both sides of the story of database and storage monitoring. I would like to introduce Jared Hensle who is the Product Marketing Manager for our storage monitoring and management product, which we have cryptically named Storage Resource Monitor.
Hey, Jared. My name is Rob Mandeville and I'm the Product Marketing Manager for our database performance product, which is equally cryptically named Database Performance Analyzer. So not really sure how much more transparent we can get than that with our naming conventions. Today--Actually, let me get to the next guy bear with me one sec. Okay so today Jared and I are going to discuss some of the contention that can be seen between DBAs and storage admins. We're going to do this by taking a format of asking each other questions from the other discipline's point of view. And then trying to answer them from the opposing point of view. So in essence, we're each going to take a turn being on the hot seat--one playing the interrogator and one playing the interrogatee. These may be questions that you as a DBA or storage admin have asked or maybe wanted to ask your counterpart. These are not comprehensive. I know that there's a lot of questions out there a lot of reasons why DBAs and storage admins may feel some conflict, some tension. So if there are questions that you have that we don't specifically address in this webinar, one of our hopes is that you can at least start a dialogue with your counterpart about those topics that are near and dear to you. So even putting together the content for this webinar, it was kind of funny some of the exchanges between Jared and myself were, we're always friendly, but we could even see the contention kind of sneaking in a little bit even at this level. And we certainly aren't in the thick of the day to day of it so to speak. But we do get it. So just keep in mind that hopefully your counterpart or counterparts are likely starting from a place of good intentions that their questions or requirements are based from a perceived need or from maybe past experience. And hopefully understanding this may help ease the tensions in the relationship. Okay so there are quite a few reasons that we may call each other's baby ugly. I know from personal experience that DBAs love to point the finger at storage for performance issues. And Jared correct me if I'm wrong but I think there's definitely a storage bus out there for DBAs to be thrown under as well.
Uh, absolutely. And the last product I actually bought before joining SolarWinds was DPA, because I was tired of doing hardware refreshes for the database guy when I kept telling him it was his database, his database, and finally used the right tool to prove that but that was after quite a few hardware refreshes because I was thrown under the bus. So yeah, there's a little animosity there. [Laughs]
Most definitely. Okay so just, that's just to say that there are many reasons for the friction to kind of maybe enter into this relationship between DBAs and storage admins. One disclaimer we are not professional counselors. So we don't have all the answers. If you do require professional counseling between the different counterparts, we're not the ones you seek. All right so with that disclaimer taken care of, just to go through a little list of some of the reasons why some of this friction can enter into the equation. It can be perceived that DBAs want, want, want. Databases in general do tend to be very aisles intensive. Especially compared to other applications. Storage costs a lot of money. In fact, for many IT organizations that is the number one line item. Storage admins talk in IOPS and latency while DBAs tend to talk in reads and writes. So we may be speaking slightly different languages. They both talk in capacity and performance of that storage, but do DBAs really understand it? Now I'd argue some do but to be fair I would say some probably don't. DBAs and storage admins have different charters from which they're starting. We're both starting at different points. Two trains leave different stations at the same time. That kind of thing. DBAs, they're one of many requesting storage. Right, so they're just one piece in this big puzzle that storage admins have to figure out. So understanding that from a DBA perspective is important. Storage is likely on the network, right? We all have NAS or SAND technology therefore more comes into the picture than just the spinning disk or SSDs. And also, you may be virtualized, right? So if you're using a hypervisor this can complicate and muddy the storage waters even further. So we are hoping that at the end if this webinar one thing does become apparent. If you have the right data in the first place, you can certainly raise the IQ of the database storage conversation. And hopefully wonderful things can happen with the relationship between the DBA and the storage admin. All right, cool. Jared, anything else to add before we launch?
No, I think that's enough pointing out our friction points. [Laughs] you did point out the complexity of storage and databases. Like you said I think having the right tools in place can ease some of that friction and really grease some wheels on getting some things accomplished much quicker and being able to show data that backs up the choices that each other is making.
Absolutely. And hopefully if we're all kind of looking at the same data set then the discussion becomes much more of how do we address this rather than where is the problem right?
Yeah. Having the same data can be used later on during troubleshooting or optimization not just on the provisioning side.
Absolutely. Perfect all right. Well let's jump on in and I think first off we're going to put Jared on the hot seat is that okay?
Fire away. [Laughs]
Okay. So as a DBA my first question is, why does it seem like asking for additional or faster disks takes an act of God to get approval?
When you sent me that question, the first thing that popped into my head was because I can you know. Right now, I may not have the keys to a lot of things but this I do so let me flex my muscle on that. No I mean I know that's the snarky answer but definitely if you go virtualized or if you buy massive amounts of storage people are like what it's just free just give it out and it's not that simple. There are a lot of moving parts. There are a lot of requests coming in there so it's not that we want to be sticks in the mud but definitely, there is some method to the madness hopefully.
Yeah and probably being one of the top line items in any kind of IT budget, probably does get quite a bit of scrutiny I'm sure.
Yeah. I mean and I know, like you said earlier, when you add the virtualization you've now got to consider IO and latency and disk space for servers themselves. File servers, you know, the other things. It's just not that one. The one oh it's just for databases. It's, no, my whole system is virtualized and so I can't just go here you can have all you can eat and all other systems will suffer so there is that balancing act of allocating space and speed to you know a database but maybe not necessarily at the expense of your ERP system. That's the front end of it or something along those lines.
Right, right. And as a DBA, sometimes when you're taking care of the trees it's hard to see the forest.
Okay, so my next question, why do I need to fill out 20 questions in a storage request? Can't I just give you my requirements or needs?
You know I would say because what I'm looking at; it is a balancing act of first can I make it fit? Hey, you want two petabytes of storage or two terabytes or whatever you want? Do I have enough raw disk space to actually give that to you? So in the screen shot provided here, I am using our SRM Storage Resource Monitor to look at all my storage between all my vendors. So you know, I may have the most amount of storage on my older EMC array, but you're asking for, ‘I need it to be this fast’ or ‘I need sub-millisecond or five millisecond type of latency’ and I can't provide that on this array. You know. So it is a balancing act of finding the correct combination of speed, space and then also like you said earlier, was the read and writes. I mean that does come into play because figuring out what the back end raid configuration or the known flash. What their dedupe and compression, what the back end is that could support that type of stuff--because if I've got your 20 set of questions and I slotted it the best I can, at least now I can, you know, clean my hands off and go, you know, you had 20 requests I was able to meet 18 of them. Here are the two I could not meet because of X, Y, Z limitations. And at that point, I feel like I've met my SLA or going to meet my SLA-- that down the road you don't come back and say, ‘Hey, I asked for two terabytes and you only gave me one.’ And ‘No, clearly here's one. We gave it to you and we put it in this particular configuration because that's how we could.’ So a lot of it is just trying to see the holistic environment and square-peg-round-hole-type scenario. Trying to get the best fit as possible.
Okay so there actually is rhyme or reason behind those questions, not just because you can, right?
I don't want to say maybe questions 18, 19, or 20 because we can. But I'm pretty positive that most of these are related to actually something working and actually trying to get the best placement. Because, you know, you guys want—‘I want the fastest stuff and I want all I can eat.’ That's just not possible. So, you know, having the best information available to make the best choice is obviously going to be--hopefully, alleviate headaches down the road.
Fair enough. Okay, so next question. Why do you keep talk to me in terms of IOPS? How does that really translate into something meaningful to me as a DBA?
IOPS and latency are king when you are looking at storage and it is a tradeoff. You know you can have desktop type hard drives, 7200 RPM Seta drives with little to no latency. But you're basically having little to no IOPS on top of it. When you want to do a database move and push a couple hundred IOPS, now you're causing delays and your latency will go up so it is a balancing act. And this is, again, what I see. This is what I'm getting judged on and really this picture, the first picture's kind of, the first line item you can see is the extreme IO array which is a AFA, an All Flash Array, you know I'm pushing 4200 IOPS but with no latency. All right cool I'm going to do that all day long that's a super-fast array but you can see the other ones I'm not pushing anything. Obviously, this is a demo set-up and a demo screen shot but essentially, you would see latency come into play in order to facilitate those IOPS. So that's kind of what I'm seeing. You know you're going to see that across all the vendors. They make that usually available through all their APIs or their SMIS connector. Some of your new arrays you're not going to see all of that data so when we're looking holistically across a multi-vendor environment these are common numbers that can be applied across multiple vendors to make the best decision. And really that comes into also from a learning point of view is as a storage admin, I may set thresholds on IOPS but really what I'm concerned about is latency. And users are going to call or tickets are going to be erased when latency is high not when IOPS are high. So once again that's something that would apply even to the virtualization layer, other applications, operating systems and your database is when latency is high that is when my feet are being held to the fire because performance is bad. Your all-flash arrays are going to push hundreds or thousands or millions of IOPS and not have any latency where your older devices are going to have lots of latency with little IOPS so ideally I just want to make sure that I've got things provisioned correctly and have something I can measure and report on.
Yeah and just to add to that a little bit databases being a fairly big driver of IOPS can play into just the overall volume that the underlying storage capacity can handle as far as you know max number of IOPS goes because is it correct that different vendors kind of sign up for different SLAs or whatever regarding IOPS? Depending on the technology that's purchased?
Yeah absolutely and real quick a question came in. Is the screen we're viewing right now, this is our Storage Resource Monitor, SRM. So this is a performance dashboard in there and back to your question Rob yeah, you can totally see we actually had it where somebody came in and said hey I bought a pure storage device and I should be able to push one million IOPS and your software is incorrectly reporting this. Blah blah blah. And they thought it was us. And actually, a sales engineer got in there and said you're absolutely right. You bought a pure storage array that could, and I'm using air quotes here push one million IOPS if configured this way. It's configured something differently. That's why you're not seeing those numbers. It's not our software it's truly your hardware configuration was not specked to push that type of stuff. So you know latency's just kind of the end all, be all, a great barometer of when things are starting' to get slow.
And that's really taking it from the engineer's point of view because they don't, you know let's face it at the end of the day they don't really care how may IOPS we're pushing' through the system. They care if they're late to get to lunch.
Yep. You know they want to process an order or view a report as quick as possible. They don't really care about the backend. And the ticket you're going to get is X, Y, Z website is slow or something vague along those lines.
Right okay definitely fair. All right so my next question, as a DBA I've read up quite a bit on different RAID strategies and different things to watch out for from a RAID standpoint. So what should I know? Or what should I care about from a RAID strategy on the storage side?
I want to defer to a fellow head geek, by the name of Thomas LaRock that I have been in hearing things, oh it's RAID five that's the problem and I do believe there was the last lab that he did that did point out that you can't put a database on RAID Five. But it goes back to obviously certain RAID configurations are not going to support the read/write that you need or expecting. So if you have a mission critical ERP system. What? If you have a mission critical ERP system or heck even the SolarWinds database that has lots of writes, I don't want to put you on a RAID 5. I know that's probably just not going to be the long term best strategy but if you're telling' me I need to house the accounting database from 2000 that's an archive and you're just going to read from it every now and then to validate some data or the accounting team is. Then by all means, RAID 5 is perfect. So I want to make sure that the RAID strategy is optimized and I'm not wasting disks. I don't want to do RAID 10 over and over and over again and cut my actual spindle account in half when you've got a bunch of stuff that's a bunch of read data. You know I'm going to go RAID five or maybe RAID six and get more storage bang for my buck at the cost of some through put but definitely I've got some redundancy built in there and this a archive database that is barely used. By all means, I'm going to throw it on RAID 5 and mosey on down the road.
Yeah so as a DBA really truly understanding the intent and the profile of how this database is going to be used. If it's going to drive a lot of write operations versus read operations. This probably goes back to those 20 questions that we talked about earlier that making sure we understand really, what our requirements are. What's our profile going to be so that we get to the right spot? We get to the correct storage in the first place.
There is. Even before giving database, you guys or even provisioning the storage, we've got to figure out a general scheme of how we're going to carve it up and how we're going to, the tradeoff between actual quantity and capacity versus through put latency and IOPS. Okay hey I'm going to carve up some RAID five and it's designed to house the company picnic photos or archived--
And I just need a lot of, I need some redundancy built in but I really care about volume. But then I'm going to have this RAID 10 or this flash and so I'm not going to have petabytes of storage but I'm going to have terabytes or hundreds of terabytes. But it's going to be uber-fast. So we'll do that in conjunction with prior to, you know putting the workloads on there but once you make it RAID five and you're putting data on there, it's not easy to go okay let's scrap it and make it RAID 10 without doing backups and recovery and all that stuff. So, setting RAID types is more of a permanent type solution.
Yeah, yeah. And just for the DBAs on the call. For databases that are write intensive, for RAID five you definitely incur that parody calculation that occurs and that's what definitely can slow down your write operations for, I've read several articles about it, multiple articles but a good number rule of thumb, probably about like 60%, 70% or something' like that. With that parody calc. So that's the overhead or impact of RAID five on write operations.
Yeah you know but if you're reading it, you're not going to see that at all. So going back to that archive database by all means, you know if you just need to validate some payroll information from 15 years ago, I'm pretty sure RAID 5’s going to be sufficient.
Yep, yep. All right, cool. So moving' on to my next question. If I can. There we go. So we've got all these different file systems and ways to format that. And you're my storage guy right and we're still talking storage because this is really how it's presented to me as a DBA. So what should I know as a DBA?
It first depends on what type of storage I'm provisioning. If it is block level, I really don't care. It is now use the database admin and system admin to go hey I'm doing a Windows operating system for Sequel. I'm doing something on Lennox. So really, that is going to dictate your file type and at that point, you're having a conversation with the storage administrator, which very well could be--I'm sorry with the system administrator--which could be the storage guy or closely related. But overall, I don't care how you're formatting it. Now obviously, if I'm provisioning at a NAS level I do care. You know I'm not going to give you NTFS and you're connecting to a Lennox box or vice versa. So I would need to know hey what is the database and the underlying OS to make that decision if provisioning from a NAS point of view. But I think most of that is kind of three way conversation with the system administrator just trying to figure even the underlying operating system architecture.
Excellent, yeah. And this question was also kind of intended to help everyone understand what different roles, different IT professionals play with in their organizations. And kind of who to talk to when. So if we're talking just provisioning block level storage, that would be the storage admin. When we're talking' about file system formatting, things like that probably the system administrator.
You know and SRM and there are significant number of storage or vendors and or storage admins that are doing NASs. So they would have input up. Okay hey this NAS is provisioned for Windows type systems and here's my format and here’s my block level where okay this NAS is for the Lenox guys or you know whatever it is so. It does come into play. But I definitely think, at this point, you probably do have that systems guy kind of running in between, putting out some specs too because they may be involved. Saying, ‘Hey I'm going to carve up a virtual system with these type of specs and just give me block level or no we're just going to attach it as a log drive or a D drive or something like that. We've already got the OS and database in place. We just need more storage.’
Fair. And just for a real quick FYI for everybody on the call, if you're a DBA and you're coming to this from a DPA point of view or if you're a storage admin coming' to it from a SRM point of view, SolarWinds also has this server and application monitor. Probably most of you know about that but if you don't that can help bridge the gap between your storage servers, applications and the database. So just throw that out there.
Yeah no, no. I definitely agree. Having those tools that they themselves speak the language or they can translate because you do see it from a storage point of view is that noisy neighbor perspective. Where I may give you a carve up data to do this but I also carved up data for some other system off to the side and it's going crazy, our backups running and it will affect it. Could possibly affect what I gave you. You know so having those systems that link and tie databases, operating systems, the storage together makes that long-term ownership much easier.
Yeah and when you do sit down to talk with all three disciplines, whoever's filling that role and sometimes people wear different hats but it just makes that conversation a lot easier right?
Perfect. Okay so in the interest of time let's get on to my next question here. How can I tell what file systems or volumes I'm depending on?
Great segue from our last question was, you know, having a system in place that does link that for you. To use the term breaking into silos, that you don't have a database guy looking, okay, ‘All I'm looking at is my database.’ And a storage guy that all he looks at is, ‘I'm going to log into the EMC or Viper and all I'm going to look at is this having a system that will tie it all together.’ Because, at a database, you want this type of throughput, but I have to consider the operating system. The operating system is doing other things besides just storage. There's CPU and memory things that come into play. So having in this case a particular, the screen shot we're showing is the server and application monitor and on the right hand side you'll actually see the server, to the database, to the host to virtualization all the way down to the storage so you can see how they are linked. This particular software is dynamic because once you do virtualized, you could have a VM where a Hyper-V doing vMotions or live migrations, moving stuff around in a disaster scenario or trying to optimize. So what you requested and what I provisioned should still be accurate but it could've also moved depending on what options you enabled in your hypervisor potentially. There's no longer one to one hey it's a physical box this is it. You can touch it. You can see it. You can do anything you want with it, you know, with things being virtualized. The good and the bad go with it that things can get moved around and no longer what you originally planned.
Yeah and this brings up another really good point is that a lot of times the sys admin or application owner or maybe even the DBA might be the first to hear about end users experiencing some kind of latency or pain. Some kind of applications hanging or won't respond, or something like that and if we know this view of dependencies, if one of our dependencies is having problems, yes that could very well be the root cause of why my end users feeling the pain. But if I can do all amount of searching through my database tier and not find the problem right? Because maybe the problem isn't at the data. Maybe it's further upstream.
Absolutely. And if I'm giving out block level storage and I didn't have your 20 questions that I made you fill out before, if I just, you know if someone says put a request in hey I need 500 gigs or a terabyte and all I did was okay here. Here it is. Do what you want with it. Without a screen like this I may not even know that it's a database so that if I go to do some sort of firmware update or I'm doing something to hey there's a drive that went bad and it's doing a raid. It's rebuilding, I may not know who to notify or what applications are being affected that it could be the backup system or it could be production. I just gave out a block of storage so this really does make it--puts all the owners at the table so you can see everything and how it's connected.
Yeah, fair point. Let's you see who your customers are as well. Internally.
Excellent okay. So right now, we're going to take just a couple minutes here and I would like to open up some polling questions that we have here. So let me open the poll. So those on the call if you wouldn't mind chiming in here. I would like you to rate your DBA/storage admin relationship and kind of how good it is or maybe how bad it is so if everybody could take just a moment and kind of vote on that. Looks like the early results are lots of sunshine and bunny rabbits, which isn't bad.
And don't use Rob and I's [Rob laughs over Jared] to translate to your personal relationship with DBAs or storage admin.
This may not be representative that's right. [Laughs] Okay so I think the bulk of it's coming in as mostly sunny with occasional clouds and maybe hamsters instead of bunny rabbits.
That seems about accurate. It's like everybody gets along. When the basketball team's winning it doesn't matter how many points you score but as soon as you lose everybody's pointing fingers at one another so that seems like option B to me.
Fantastic yep I think that does make sense. Okay so now I'm going to open up our second poll here and that is, let's have everybody vote out there on which IT role or roles, so you can't answer select multiple things that it could cause the most angst within your kind of day-to-day life.
When you threw Dev Ops in there or development that'll be interesting.
Yeah well, I mean it's fair, right?
I think they live by the mantra do what you do.
It worked in test.
Worked on my dev box.
Yeah and that's to say that a lot of times, especially in a bigger kind of IT shop, your DBAs may be broken out into development and production and a lot of times your production DBAs just inherit the code that gets running in your environment so they don't actually create it. [Speak over each other]
That the multi-select wasn't working or maybe I misread some sort of pop-up.
Okay you know what if that was the case I apologize that was supposed to be a multi-answer one so I guess if that wasn't working people probably chimed in with the one most of interest to them. So interestingly--
I'll just make the assumption that DBAs would've been selected a lot more if we had multiple selections. [Laughs]
Or the storage admin either way.
No, no, no DBA. [Laughs]
But actually, the development and engineering was the clear winner of that poll.
Definitely. I think that's a joint enemy.
I agree I think so. Okay well thank you everyone for taking the time to give us those results for those polls and now we'll continue on with let's put the DBA in the hot seat.
All right let's flip these on you. How come you as a DBA, you tend to ask for massive amounts of storage from day one? Asking for a petabyte or terabyte you know, and I know that this is a new system. How come you don't know what your expected growth is going to be or grow it out?
Yeah very fair question. Probably my best answer is that we very rarely get good specks from the application owner or development and don't even get me started on third party vendor applications on storage needs or growth projections. As a DBA, I've definitely been bitten many times in the past where I've had to keep coming back to the storage guys and begging for more and more and fairly quickly, right? Which puts you guys under pressure as well because the worst thing I can do is have my systems run out of storage because then things are down. We're going from needing more storage to an actual outage and then everybody's in emergency mode. So by saying that as a DBA it was always nice for me to kind of remain the master of my own domain to stay in control so I would always like to have some storage in my back pocket if you will to make sure that I don't run out. So I might build a little contingency in there for myself. Now after the systems been up and running for a couple years, you know then I can definitely speak a lot more intelligently about the predicted usage and growth patterns and things like that but until then this is kind of a CYA play for me. I need to protect myself. If I do run out of storage then who's going to get blamed? It's going to be the default blame acceptor, which is the DBA. [Laughs]
Yeah I will say there are, we have talked to storage individuals that their sole job was just capacity. I mean that is what they did was ensure there was capacity so to your credit giving us worst case scenario, that buffer okay I may not like giving you all that storage but in my mind I gave it to you. You better not ask for anymore and I can continue on with my plans of asking for another more storage down the road or whatever. I'm not anticipating you calling me up saying I ran out. Give me more. I've moved on and now I'm trying to meet my SLA and ensure I've got storage capacity for other needs.
Right and you probably don't want to keep going back to the whiteboard every time somebody runs up to you and says I need more storage, I need more storage. Because you probably do capacity planning a few times a year or maybe once year when budgetary cycles are in play.
Yeah. For somebody to keep kind of moving the target or moving your cheese, that can't be a good position to be in.
No don't move my cheese. [Laughs]
All right, cool. Next question.
Okay obviously you're coming to me and probably one of my 500 questions I'm asking you is you're asking for super-fast storage but can you prove to me that it's not CPU and memory are not the issue. That it is a storage performance issue. If you want me to move you from a traditional net app or whatever, it is to flash or something super-fast.
Right fair. So and this is kind of cavalier but my fist response back is can you prove that it's not. So kind of a guilty until proven innocent kind of thing. So if I have a good system in place. What you're talking about what you're requesting from me as a DBA should absolutely be possible. However if I'm kind of relying on running my own diagnostic scripts or doing my own trouble shooting using these queries that I've come up with, it's sometimes very difficult because once an event has occurred, you can't go back in time to see it right? To see what the root cause might've been. So we do need to be able to troubleshoot real time. But the number of cases where that comes into play is pretty rarely. In fact usually it's hours. Sometimes even years after an event actually took place before I hear about the issue. So being able to have something in place like a monitoring tool that is watching 24/7. That can go back. I can usually go back and kind of do that post mortem on issues with is very key for me. Because then I can prove that yes it was maybe a memory pressure issue right? With page life expectancy things, kind of aging out of cache. Like you can see in the lower right hand corner and that was driving a lot more I owe than I normally do during those periods.
Gotcha. And then I want to also say, all these screenshots you're showing are now out of a DPA correct?
Some are out of DPA some are out of the Orion side. So we've in fact with the latest release of DPA and DPAIM, the integration module, we have actually done a much tighter coupling between storage resource monitor and DPA. So we can start to bridge those gaps a little bit better. So these specifically are out of DPA you're correct. But some of the screenshots that I will show are from the Orion side.
Awesome. Let me ask you this next one. You know, you're asking for all this storage. Do you have an archive plan in place going back to my tiering of RAID and moving things down a RAID level or you expect the database to grow and grow and live on forever?
Yeah, great question. You know my simple answer and again kind of a little bit tongue and cheek here but simple answer's no. Right? When we're rolling out new applications or databases to support those applications I don't think we think too much about archiving plans or some kind of futuristic three years down the road, four years down the road. When it comes to our data modeling so the same thing with third party applications. Sometimes they will have a mechanism where we can archive and really, what you're talking about is probably within the data model structure itself. That I can move some of these things off to lower tier storage to take advantage of maybe things that are read only and not so write intensive things like that or that are going to be accessed very infrequently. So this is something that may come into play later down the road like I said some futuristic thing. May not even be my problem by that time so why worry about it. And the other thing to mention here is excuse me, a lot of these applications go to support specific functions or maybe my company is within a specific industry. Which has a lot of compliance and rules or regulations around it that says I have to keep this data available for X amount of time. So sometimes, I'm handcuffed and I can't really even impact that too much. So as a DBA, I think probably my safest answer is I'm just going to go with grow and live forever.
Well to rebut that I said archive I didn't say delete your data. I know from a compliance standpoint you're just going to be like all right I'm deleting all of 2015 go. But definitely going hey can we move 2015 to run some mechanism within the application or database and move it over to some other storage device to keep you happy on your current storage would be ideal. Plus just from a pure volume standpoint. If you've got maybe speaking out of term here but if you've got 10 years’ worth of data, your searches are going to perform slower than if you've got two years’ worth of data in there.
That is fair but if those kind of decisions or those kind of thought processes aren't taken into consideration when you're doing the initial development or data modeling, design then it's very difficult to get there after the fact. Your talking about migrating data and this is something if well thought through from the beginning yes should not be a problem. However, to kind of change your horses midstream, that becomes a different proposition.
All right. If I give you all flash storage. Can you insure that the database is optimized because as great as AFA and that is, it can hide some—jus--issues. You know it just can mask them by just giving it more IOPS and more latency where really it may not have been necessary because there was something in a loop. It just gets out of the loop faster now.
Yeah fair and before you go through the point of an expense of doing that, from a DBAs point of view since we are IO intensive, it is good to start there at east limit my exposure to any kind of disk latency. I do realize that there's lots of other factors that a DBA needs to consider. Kind of like the following like database parameters and efficient inquiries, indexing data model et cetera. But you know nowadays with the relative low cost of fast disk, why not start there? So this may actually buy me some time to focus on other contributing factors. Now I say that understanding the caveat that we can't always just feed the beast right?
I mean yeah I don't want to just move everything to flash and at some point, you’ve got to pay the piper. If you put everything on there and everything's written poorly, you will and can see performance segregation at some point. It does not scale indefinitely.
Yeah, yeah. Just kicking the can down the road might save me temporarily but eventually it's going to catch up to me.
For sure. And also speaking to the general context or theme of feeding the beast, if you start to do this with all resources, not just storage, so if you start to think about memory and CPU processing, you can actually increase your licensing costs from a software perspective. So it may seem like the right answer just to throw more hardware at it but you’ve got to be cognizant of that, because you may actually be pushing your company into either liability exposure or certainly more cost down the line potentially.
Absolutely. Making sure that getting the most out of what you have as opposed to just giving it more and more will come to bite you at some point.
Um, okay. I know somebody hit this up on chat. I guess there's chat earlier was can you provide me with read write numbers for your database so I can go back to that array question; provide you the correct array type. I mean if you run, I know--what was it--Dell D Set, you--Now, I used to run that stuff on just servers in general. To say this is what their workload is and I'm sure you can do it from a database perspective of hey, it's heavy read or heavy write, then obviously that does come into play on what array I put you on.
Yeah fair. This really does come into play with how often if I'm using custom scripts myself then this really can come down to how often I'm going to run these, how granular the data is that I'm going to capture or if I'm using another tool to monitor and collect this kind of information, how granular does it get? So yes, I can give you some read write numbers from maybe a very holistic point of view. Maybe I run them daily. Maybe I run them hourly. Don't want to run them too often because that's put additional load on my systems. So but if you're asking me throughout a normal workday what would be my pattern or my usage kind of workload throughout the day, that gets a little more difficult to tease apart the numbers. Again, I can tell you from a daily point of view or maybe from an hourly point of view and that may be close enough. That may be fine grained enough. But also that's more work on me right? I've got a lot on my plate. A lot to do as a DBA and me making sure that you understand my usage patterns from a database point of view that's something that's probably on my back burner to be honest.
Well you know I know it's always about the DBA. I'm using air quotes here again but help me help you by, if you're able to at least go, hey this is what--if you gave me some hourly or even just time of day I could tier the storage. Like hey, I can put these two databases on the same storage because this one is used in the morning for, you know it's maybe a maintenance or some sort of shift. Hey, I've got a morning shift and an evening shift or some sort of calculations where hey, I can split this workload in half. I've got some happen in the morning, some happen in the evening and if you can show me that then I'm like oh I can put that on the same storage device to help you know get that load in the best spot obviously all the work loads are all happening simultaneously. I'm in a pickle but if I can figure out maybe some time of day stuff, I can tier it according to that as well.
Yeah fair. And that's where having the right kind of monitoring from that aspect that can do that mapping that can kind of bridge the gap between the databases and storage can really help out tremendously. And actually here in the screenshots I've got the left half over there is pulled from the Orion side and it can show the volumes and the different pattern profiles really tends to drop off at about eight AM things like that. And then on the right hand side from DPAs point of view, this allows me as a DBA to drill into very specific queries and find out which ones are actually driving these disk reads and disk writes. So really helps me answer the question from your perspective what is my profile? What are my needs? What are my requirements from an IO perspective? To me being able to understand as a Database Administrator hey what's actually driving all that workload?
Gotcha. Actually going to the last question was, do you have past usage data so I can make the correct placement? Avoid those nosy neighbors. In a VDI environment you know I'm going to have massive amounts of read and write at 8 AM when people are turning on their machines and booting them up you know. So if I had some sort of database thing I can take that into consideration of my other systems that are using that back end storage.
Yeah absolutely. And a fair question for sure. Now for me to do that individually or for me to do it for my DBA team or something like that that's quite a bit of work and maybe impractical if I'm going to be just using scripts or maybe a monitoring tool that doesn't quite give me the information that you're requesting. So again with the right kind of monitoring I think that we can get some really good mapping some really good understanding of first of all the dependencies of the storage that is used by my databases. That performance summary. So I can see by storage volume there or by storage file system exactly what's playing through there. Throughout the day so I can look at IOPS wait and see through put. Things like that. That really helps me get a good profile or view of what my systems are driving through the infrastructure. And then again being able to take that information and then as a DBA be able to answer the why or the what is really key. So I know that most of this IO is being driven from a specific query. I know that I'm doing a ton of disk reads and now I know why. It's that this query for probably data modeling reasons is doing full table scans of all these tables right? So maybe I have to go review my indexing structure things like that to help reduce the load on the infrastructure. So here, I can help you help me by helping me. [Laughs] I don't know I got lost there. [Laughs] All right. Next.
Um you know let's just assume this is a virtualized system. Do you have, are aware of any affinity or anti-affinity rules that would limit your VM placement? Because obviously I can't create storage in the west coast data center if your application needs to be in the east coast data center or something along those lines.
Yeah and this is where as a DBA probably those discussions were had But it's probably not at my level. So that's somebody probably a little higher up--that's doing these licensing negotiations and putting contracts in place and stuff--would probably likely be asked. I know that there can be issues with basically where your databases can run which processors they have run on or could potentially run on in a virtualized environment. So you do have to be cognizant of that. But again, that's probably above my pay grade.
Yeah that's getting the systems guy involved you know essentially you may not want to put all your accounting databases on the same array because if that array went bad then you potentially could lose all your accounting databases. So there is some of that minimize catastrophe by making sure that certain workloads do not sit next to one another or even from a virtualization host perspective. That if one host goes down it doesn't affect. Everybody was on there and this whole system goes down. Spread out the load that kind of stuff.
Well then I'm glad then I kind of stumped you on a question or at least you didn't come back with some rhetorical answer so.
I can hand it off to somebody else so that's good. I like to be evasive. [Laughs] Okay so before we get into any Q and A's I was just going to kind of summarize a little bit. Some things that can definitely help raise the IQ of database administrator and storage admin kind of conversations, make sure you have the mapped visibility and common ground. So really, what I'm talking about is the understanding of the storage dependencies for each of the database instances. Is a really good place to start. If we understand that dependency tree, then we know which parts and pieces of our infrastructure are actually coming into play. Getting the visibility into the storage performance definitely breeds more trust on the DBAs side and probably on the storage side as well. Again, if we're all working from a common set of data something that we can all agree on, this is what's happening in our systems. Then that makes the discussion between the DBA and the storage admin that much better. Okay. Knowing which databases are leveraging which volumes and which ones are driving the IOPS provides great visibility for the storage admin so that they can actually self-help a little bit too. They can see what's driving most of the workload through their systems. They know when to go talk to the DBA about it. So if we have this clear picture of this dependency mapping it really helps the discussion when performance issues happen or maybe when capacity questions come up. Again basically, we know where to look and we know who to talk to at that point. So if there are storage space limits, the storage admin and DBA can talk strategy. To deal with it and by this I mean things like maybe log rotations some stuff that Jared mentioned. Maybe archiving data. Maybe doing some storage starting or even application charting from a database point of view. Prioritizing performance needs. Et cetera. Do we have our storage in the right spot? Which is what we alluded to earlier. If your databases are driving huge IOPS, the DBA could help identify the cause. Things like memory pressure. Things aging out too quickly, out of cache. The application logic and round trips. Right? We can look for opportunities to cache things at the application level rather than having to go back to the database each time, which may cause some kind of physical IO. Transactional commit logic. This comes into place where you know within our application code or database logic if we're committing after every single row and let's say a large update, then that can cause a lot of pressure on the transaction log. Versus say batching. Like we want to commit at the end of all these updates or maybe after 10,000 records, we go ahead and do the commit. So drive a lot less workload through our infrastructure. Storage admins should have a good handle and be able to explain that the cause structure and impacts of DBA request. Right so if we come to you with a new database that needs to get rolled out in support of a new application or something like that. Being able to talk openly and transparently about that, about the impacts that it's going to have within the storage tier than I think just with that transparency and with that open communication and dialogue comes a lot more trust both ways. And that should help make better decisions on both sides of the aisle. Okay so do we have any questions from the field? I'm looking at the Q and A; I'm not seeing any.
There was one that came in about on the Orion side for Net app storage on tap non-cluster mode. Yes, SRM does support that. It was just the array type. So and then the latest releases that SRM provided was for pure and extreme IO. The flash storage came out a year ago as well.
Okay. Perfect. Okay so in the interest of time I think we're coming up to the top of the hour here but on behalf of Jared and myself I would certainly like to thank all of you for taking sometime away from your busy schedules to step away with us and hopefully have a little fun today. We definitely hope that this will spark a conversation with your counterpart or counterparts that can help you come to a better understanding of each other's roles, priorities--kind of what drives or motivates them, and what they're starting from in the first place. Kind of what their charter is versus yours. And to go back to my disclaimer if you think you need professional help, please see your director or VP of IT or whoever plays that role for you. You may not get the help you need but at least now, you're going to have new common ground to talk about that's for sure. And fourth and this is really just for the DBA side. Go ahead and continue to blame your storage guy until they can prove otherwise.
Wow, wow. Way to end the call.
I had to throw that in there. [Laughs] But I say that in jest of course.
All right, thank you all and have a great rest of your day.