New 130TB Storage Server ft. Linus

New petabyte project.

We ran out of storage again, so we have to temporarily add these 512tb SSDs for more space.

Next video can explain how he backs it all up. He does backups right?!

At this rate, Gavin's going to need a server room.

Linus Tech Tips PETABYTE

Linus needs to give some further tips on RAW video versus intermediate (archival) codecs versus delivery codecs.

What's funny is that he uses way more than that himself so its shouldn't have even been a surprise

Steve Thea thank you very cool

@Brian Egendorf I mean. It's whatever the sponsors will send over. If it was up to him he'd be rocking an exobyte in his server and giving out petabytes.

@Brian Egendorf Well keep in mind the method of storing the data can affect how much data is used. You can substantially decrease the amount of data required with no noticeable loss of quality or usability. Storing data for archival properly can do this.

I cant like this comment, it would be a shame

Then Gavin "That's not the way I would have done it"
Then Linus "Me either. Usually I drop it completely"

@@RipleySawzen WHERE I CAN GET GAVS COOL SPACEX SHIRT 8:44 ???

It's Linus, what do you expect?

The dent was on the opposite side of the drop.

That astounded me. Almost as shock-inducing as *beeeeeeeeeeeeeeeeeep*

He has said it before... There are a few videos where he's mad or where they look at bizarre/enterprise tech.

He just don’t want to think about it that late of a night.

The magic of Gavin's question asking at work.

I bet he actually just didn't feel like explaining it.

Your comment made me laugh harder then when it happened in the video. LMAO

Gotta entertain yourself while he’s setting it up.

That ant he found is a queen, shes going around looking for some soft moist dirt or wood that she can make her nest in and start a colony. The footage isnt that great but my guess is that its from the Camponotus genus

That made me laugh, just because it's a really small ant. We have things called "inch ants", named for their approximate length.

@@Jaybiiird AntsCanada? Also to be fair, it was a massive ant.

Actually, LTT made a video a few months ago about how they have completely filled up their petabyte (1000 Terabyte) server.

@@Steamrick Yea I remember watching them struggle with that lol. But per second the phantom is still hogging more storage. plus LTT putting out a lot more videos of course

Yeah, per second it's pretty much impossible to beat a professional slow-mo camera. Unless you count the complete sensor input from the Large Hadron Collider or something like that, I guess.

@@Steamrick Which is by the way, 25 petabytes per year, and less than 1% of it ever gets looked at.

@@Steamrick technically it's 1024TB in 1PB

The biggest that would be possible*

😂

You'd think it would be the CIA requesting this, but I guess not.

@@ashtonhoward5582 We need something to play in the break room

You guys have 150TB and it only lasts for 3 months
Me: has 200gb lasts for 5 years now

I think you have already seen it...

best part is gavin understands it all. his brain is just fried because he was apparently awake for 19 hours from this video.

@@Zambrano2199 I could imagine Dan here just dazed

Linus: Hack hack hack. Sorry....

damn thas crazy

@@chrismofer 2

all those tapes.

Been online for 90 days..... already almost full...... "Yea i'll make it last 8 months"

Just like his flipflops

shinigamidragoon Well he does film almost 12 months of released footage in those 3 months so it might last him 8-12 still

Tech guys are always dying inside

Blake Rook Official damn..

ahahah this is too funny

Yup, he looked shook upon hearing how much data Gavin and Dan are consuming.

@@Johnlanzer afaik linus is consuming even more.

@@lusteraliaszero Don't tell him that or he'll lose his mind over how much hard drives and custom made server farm will be needed.

Gavin is the Chrome of hard disk storage.

@@Johnlanzer Yeah, Contary to popular beleif Linus is very responsible.

it was actually only online 90 days before it was almost completely filled 😂

@@Vykori Uhhhh... 90 days _is_ 3 months.

@@DasVERMiT Wow you're right. I'm stupid. I was thinking about it like hours, 90 minutes is 1.5 hours. 🤦‍♂️

@@Vykori seems like linus needs to make a new sever for gavin lol

Well Linus has a Petabyte, and I'm pretty sure he has almost filled it.

😂😂😂😂

Why is this so true😂😂😂

No Linux tech tips more rude

OOF

Gav is always the cool guy in the room

The Red breaks the clips into 4gb chunks.

@@thatvideoguy93 The files break into chunks, but the camera keeps running.

That was to create a test

All of storage. Now. You have 3 months to write this data to disk.

8.5 quintillionpentabytes should last him 1month

Damn

Don't worry, un a few years linus will be making Yottabyte servers that last a couple weeks

Its like the Internet Cold war with who can fill up the most storage space

@@jonbrandre3006 WD get to work 😂

I got 1 tb hardisk and already full in 2 month

User Bots . I’m sorry. That was a month ago. I got a 2tb hard drive and it is almost full with videos in 1month.

I thought I never needed more than 32 gb but I got full quickly

Ive gone through 7terabytes in one year

I had 8 gigs on my old phone, which I had for about 2.5 years

I think what Gavin needs is Petabyte Project pt 2

@@Rich-gd4fc yes the slow-mo guys could legitimately use the most storage you could throw at a server.

Why doesn’t he delete the files after he’s made the video

@@Jonasdrady That's what KRplusrs used to do, but they're learning that they may actually need that original footage in the future, so now they try to keep it all.

I think he meant to say hibernate

Always go for the 60 drive unit because the drives make it really expensive.
And you can just add drives as you go.

more organs means more human

@@poochyenarulez he deleted his page file so no

wtx quality content

What if you don't have an attic.....

@@jakobfindlay4136 He'll use your closet/ crawlspace

I've said that like 300 times now, but still have not even a single 130TB storage server in my attic. :( It was worth a "few" tries though!

Don't put servers in attics.

Are you freelee?

Correction ...
Linus: How long until you fill 1 petabyte?
Gav: 1 camera

Ha Linus didn't think of raw file storage, he may not use his cameras "like a pro" therefore never realized.
Okay a frame rate... Per second.
So 60,100,200...
Raw is 3x (or more) the frame size due to the meta and fact it's unprocessed.
So I'll do this as stills...
A 6mp camera takes a stand image of 1.2mb.
A pro of the same 7mb.
So if you're slow Mo shooting at 100 frames a second. That's 700MB a second.
So let's get 1 minute. 4.2gb.
A hour 252gB.
In law we use a term "bound to fail" storing data is required and useful to a degree, but planning is required.

We need to get Gavin a hundred yotabytes just to last him a year!

I am pretty sure a peta is eq to 1000 tera and not 130 tb

Bruh that is fucking genius I'm actually scared.

They should just take some brains and make bio data storages... pretty sure it will get full in 3 months tho lmao

exaexabyte

Yep.

@@icantth1nkofanam40 Biggest I know of is an Exabyte which is I think 1000 Terabytes*.
*Correction, Petabytes

@@minecraftawesomeness 1000 Petabytes*

Linus* ;)

All, every of the storage ;-)

I think it's Gavin's sense of humour too :)

misomalu ‘fill her up... fill her up...’

There's the advantage of unRaid over other software raid systems, you can upgrade it drive by drive, since it supports mismatched drive sizes without wasted space. Only catch is the parity drive (or drives) have to be as large as the largest disk in the array.

WD Reds go up to 12TB. Seagate Ironwolves and Toshia NAS go up to 16TB. So yes. But if I was going to buy 15 of them to upgrade, I would also buy another server box to put them in and keep the existing one.
Above 10TB, the price per TB goes up a lot, so it is probably cheaper to buy more drive bays.

@@katrinabryce I would buy the highest possible hard drive capacity, because this can save you additional costs and space with a smaller case. generates less heat, and consumes less energy, especially in the summer when the air conditioning additionally has to dissipate the heat loss. Not everyone pays only 13 cents KW h in Germany, for example, you pay 36 cents / kwh. and it costs about $40 per year to run a single hard drive, and that's $ 400 for a 5-year deployment. This means that if you can save 3 hard drives, saves the additional $ 1200 on electricity costs.

@@fnordist Your math is a little off. $40 x 4 years is $200 not $400. Also your $40 per year number is like double the 100% read/write energy cost. These are going to run around 4-48kwh per year per drive. Given they are going to be idle or standby most of the time, I would estimate 15-20 kwh per year. Call it $5-8 (@0.36/kwh) and additional $2 to cool it. That will give you more like $150 over 5 years for 3 drives. With cheaper power, very low activity, and an environment you don't have to cool; the drop to $10 for 5 years.
My prices are a little on the low side because they don't take into consideration the efficiency of the power supply, but if you want to consider the system as a whole. If it uses 450w at full load taking out the three drives is only going to give you a 3-4% power savings.

@@barongerhardt which idiot has a 120 TB server if the server is not at least 60 percent busy? With such a low utilization cloud storage is cheaper.

Linux

Winuix

I think its pronounced ASUS

Anustechtips

@Nicolas RAGE ok boomer

Would wow ten times as hard when he says how long that takes to do
(Not long)

Let's see it's 1.5GB per second of footage. So yeah it's weirdly fast. 90GB/Minute of footage, so 900GB for 10 minutes, which is incredible...

.

@@livedandletdie I'm far from being a tech savvy person, but doesn't that mean that these cameras have one hell of a lot of power to be able to produce, transfer and store 1.5GB of data per second??

@@wisefelipe That's why they're so bloody expensive.

Yeah same here. At first, I actually thought this was an LTT episode. I was thinking to myself "damn this video is super chill and relaxing, they should do videos like this more often".

Ya I like it

Linus hyper au naturel. I kinda liked it...

The good old days of LTT before the warehouse offive :(

It's like switching from the US version of Kitchen Nightmares to the UK version.

NorteX and I thought my 1TB SSD was a bit overkill...

@@ElZamo92 We've build a 6PB cluster for a customer last year. They brought in hard drives by the pallets :D

Thomas Vleermuis lol

Every second on their video (raw from the camera itself) is 24gbs

It all camera data based. These new 4K and 8K cameras open up a whole new segment of data storage. The good news is that that type of storage is now within the realm of the common man. In a time not too long ago only a multi-million dollar corportation could have been able to afford that kind of storage capacity. Aren't you glad you live in these times.

around 22:30

idk if he's underprepared, afaik 200 terabytes is also peanuts to LTT

I did comment else where about this disaster.
You would have thought he would have considered real photography (videography) as he has similar cameras, or are they just not used correctly... Not that he needs raw data.
I saw 130tb in a old tech "sever" that would have never touched this issue at all. Here is a 2month solution... It's the wrong solution and done the wrong way.
I'd you look back Linus is not a real tech and do see his server and storage stuff & set up.
I'll say no more I find the video Linus makes for entertainment purposes only (and other) just that.
It does appear good though...

@@guywhoknows he literally uses this setup himself and he uses more data than the slowmo guys, he's just playing it up because it's funny how much data they are using. If you want to locally store this much data, this is the right approach.

@@lusteraliaszero
I would disagree with the approach.
It is kind of right for SME or general domestic.
When your talking uhd raw data is massive, even HD.
The correct "good sense" would be a main data server with an attached array. Like a SAN.
That way you can add to the array and only add storage to the network rather than adding servers with drive space rather than a storage array.
For example a small sever with few drives attached to a 48 bay array. Even if you use only 8 bays, there are another 40 to add.
Dumb arrays are cheaper IMO and would be good sense.
But you could attach USB drives to the server and swap out data.. if you don't mind waiting.
Perhaps data management as in policy is required.... Why keep raw files?

Rory Witham having done some research on why some you tubers keep raw files... it’s so they have original quality video on hand should they reference something and desire to play the clip without trying to get the file from KRplus itself.... pretty sure Gavin says that

and the text file they put in it is anus....... slow-phatty/chonky/anus.txt

@IbukiHimari Um actually ashamed yo say I laughed at that 🤣🤣.

That's a typical Gavin right there.

@@CameraManBlaise They hate us cause they anus

IbukiHimari would you like to open it?

I was thinking they should have done that instead, but this is still good

That would be so rediculously expensive probably only worth it if he keeps the channel going for like at least 5 more years so maybe he's thinking about it

@@TerraCAD nah, thats only another like 18 - 24 months of runtime if he goes at the same rate as the previous 3 months

@@calebcarr3986 1000=6.7(rounded) x 150 so 6.7x3= 20.1 meaning it should last 20.1 months. you nailed it lol

@@calebcarr3986 but he used much of the space of often used footage and this I assume is a one time thing if he makes a total switch to a server only system and i am way to high to really explain myself

you do realise he's wearing an Achieve shirt right?
you know, Achievement Hunter Merch.........which is RoosteTeeth merch.................

@@blaintintnoob he wears a SpaceX shirt in the main part

@@blaintintnoob not for the majority of the video, Einstein.

@@blaintintnoob Oh yeah I guess he is in the little end cap thing lol! I missed that thanks!

Why do people gotta be so salty when responding to people?

but still linus have to pay for all of that drives right? There no way seagate just give maybe 1 mil worth of drives free of charges to linus

@@blackgrim5151 I work for 45Drives, I can tell you that for certain projects storage is indeed given. Think of it as money they would normally spend on marketing to get their product in front of peoples eyes, they instead just give away some product and it has the same exact end result but it actually ends up being cheaper because they make the drives for much much cheaper than they sell them for :)

@@blackgrim5151 all they have to do is give a few well-known people with a huge audience the product to get their name out. like what Mitch H said. they are in a way investing in linus for advertising. who sponsored iJustin, dustin and gavin for the server? 45drives. so if I want to buy a server who do I find? 45drives. you get the picture right? To get their investment, they need to embed their brand into someone's head. When someone hears about servers they will automatically think about 45drives, which directs customers to them and that is where they get their returns for the spending.

@@mitcHELLOworld That's a smart move.

@@Syn741 Also, a few hundred drives is not that much to their production volume. Especially, not for the return.

*2020!*
Litterally a couple seconds after that
"90 days have passed, it's almost full"
Oh c'mon now.

yeyeyeyye

More like anime crossover ep.2

whats the electricity bill for 1 petabyte of storage?

Exa byte

Dave L unraid doesnt works like that . Most of the drives will be in sleeping state . When ever the drive need to be accessed . It would be active again . Thats the benefits of unraid. For 1 petabyte if he choose 16tb ironwolf drive 64+3 parity drive . Those drive are rated for 1W for standby and sleep and 9.6W during working .. so during standby conditions it would consume additional 64W during standby ,.. if he opts for 5 petabyte then it would be 320W during standby .. It would serve atleast 5 years for him 😅

I want to see a video of that build!

Typical Person i wish i could see that in real life 😅

You know he made a Techquickie video about this right?

this sounds like a crossover oppertunity

i dont know to much about ants but i think it might be a queen because of the size of her back compered to her head also wear i live ants this size are pretty common as their are more then 1 species of big ant wear i live by the way its in the south of israel

@@shikmanyam sounds more like a soldier ant

@@shikmanyam It was definitely a queen ant here in the video, I recognize it. Campo. Penn...

I used to go to summer camp with Ants Manitoba before he blew up.

at least this time he knew it wouldnt be enough lol

I really like both Destin and Linus, I didn't like that video. I hope I'm wrong, but it felt like they didn't enjoy each other's company. Their collab felt tense to me, for some reason. Hopefully I'm wrong and just misread the situation.

*days

Pointless effort tbh

This thing costs money. If they got it from sponsorship, probably they think they were getting back a whole lot from this project, so they gave the smallest one.

They can't be serious about this lasting half a year. Do they just not delete superfluous footage? Why the fuck would you keep *days* worth of slow-motion footage?

@@TheHadMatters They keep all the footage for archival purposes

1 whole minute!

Every 60 seconds in Africa, 1 minute passes

1 fucking minute!

@@PrabhablyAGoodKRplusr this is false, I live in Africa.

I think they should be able to store 2-3 hours of footage.
I know it is a joke, but i was curious.

BTop Live i read it as soon as they said it lmao

*bird noises*

"do you actually have a file called 'anus' ?"
"...would you like to open it?"

Drop? Like Linus? I'll show myself out...

to be honest having no idea, i would have assumed it would have cost much more than that

Lol. It’s way more.

@@LlamasAreBest nah, this small version is right up that alley, that's why this BS thing marques and justine bought is so bad, it's overpriced by something like 500%.

@@Markus-zb5zd A comparable Jellyfish costs $45,000! Unbelievable...
(assuming configuration: tower, 200TB (132TB usable), two 10GB ports (one NIC), no other upgrades, or even a support plan)

Probably because he filled it with the old footage (according to himself in the end of this video) he had stored on external drives (and a few small NAS').. If he deleted the files that now is copied to this (organized and easily available one), then the he technically have a lot of free storage space again..

@@C0MPUTERPHILE Oh yeah for sure, I just find it amusing, haha.

How long till Gavin and Dustin get Jellyfish servers to replace their unraid ones?

@@andy_byrd any difference?

@@andy_byrd instead of a 30k jellyfish he should get a petabyte server

@@andy_byrd Jellyfish and 45drives have totally different philosophies on how we do business. Us at 45 Drives are completely open with pricing, software etc. We have a github for anyone to comb through and use as they wish and use all open source software. Jellyfish however won't even tell you where the breakdown of costs come from. Jellyfish is for sure a decent product to use for direct attached storage for specific workloads, whereas our storinators are literally a jack of all trades. We don't recommend UNraid these days we have moved on for our recommended use cases. We use a combination of FreeNAS, CentOS + ZFS and for our clustered storage solutions we use Ceph. Cephfs is hands down the best distributed file system out there right now and our clusters can handle anything from Object storage, to file system storage, to block based storage. Not only that we can do all at once. If you want to run an object gateway like a S3 bucket, combined with SMB/NFS shares running on CephFS, as well as iSCSI block devices you can go to town! Essentially Jellyfish is decent at one or two specific things, whereas we do it all.

@@andy_byrd 😭

_conspiracy theory_

And sponsors.

Some of us do understand

@@rollover36 His facial expressions didn't look like he did...

@@Hoch134 if u know anything about computers that was like the alphabet

Adam L Still better than being an asshole like you.

@Adam L you have exactly the same kind of profile pic tho

That's Ainsley Harriot you fucking normies

what the hell happened here

@@SreenikethanI Farce right trolls from Twitter try to intimidate and fail.

Nah he definitely swears. He deeeeefinitely swears

I swear :"D

you dont say😂😂

he's still not allowed to.

IT types swear like sailors typically, from my experience. Lots of bs they deal with

Yeah... over on the LTT side of things, their Petabyte storage server is out of space. That's about 8x the capacity of the one in this storage.

@@Steamrick Yeah but they filled it up in like 2 years.He filled it up in 2 montha

@@kernelpanic2887 I mean... 3x8=24. So they both filled at the same speed

@@Steamrick bear in mind that has been running a hell of a lot longer, and they film a lot more frequently than gav does too. They've been working hard to fill that petabyte, gav probably filmed dan getting hurt a couple times

Several of the cloud providers are counting their storage in exabytes these days. That's millions of terabytes.

"Don't quote me on this"
-Also Linus

@@repzo5551 bruh lol

Eh

Mouth hygiene :D

I have to admit, I laughed way more at SLOW-PHATTY/CHONKY/anus.txt (an empty anus.txt) than I should've.

@@Cheesyxable Same xD

@@Cheesyxable lol same

Anyone notice that the anus.txt file is 2 years old... there's no way Linus was saving his anus for a rainy day or something like this