The Lemon Fool

didds wrote:If its effectively a NAS you need then I'd consider a Raspberry Pi - there are plenty of turtorials/howtos online to make a NAS from one. Its as small as you'll likely ever find currently, and has a fully controllable OS (raspbian - version of linux). On top of the Pi itself you'll need some external hard drive provision which could be as simple as a usb drive(s).

Depending on how sexy you want to get you could incorporate RAID into that etc etc for data intergrity purposes

didds

That's my next project. Here's an excellent, no nonsense guide (none of the usual youtube whooping and hollering)

https://www.youtube.com/watch?v=bpvlEbdA6qI

The problem that I see using a Pi is electrical power. Most USB hard disks are powered from the USB. Not so much a problem for SSD (unless you go daft with LED sparkles), but that gets quite expensive if you want a very big disk. Rotating hard disks take a lot more power.

Western Digial did sell some portable units under the brand name Elements that used a full size 3.5" drive and came with a power adaptor which should negate the need for anything more than a Pi.

Tesco were dumping stock a short time ago and were selling the 2TB units for about £45. I bought one and it is still in the box (I'm not trying to sell it to you) I decided that a NAS would be more useful and in the end got one of each.

As usual, you'll have to do some hunting as the Elements name is used both for the larger (external power) model that I have (about the size of a mid sized holiday reading paperback) and the small laptop sized drives in enclosures which only have a USB connector.

If you fancy the Pi route, I would expect the Pi to be able to handle an SSD and if you can whack on a big old hard drive with its own power block, you should be fine.

I know this is drifting off topic, but there has been a scandal recently on a hard drive technology called SMR or Shingled Magnetic Recording. It is unlikely to appear in a drive of under 2TB but for those with a bit of time this is an interesting place to start reading.

https://arstechnica.com/gadgets/2020/04 ... -channels/

Regards,

B.

Here is a big collection of mains powered external USB drives:

https://www.cclonline.com/category/702/ ... t/1064001/

dave559 wrote:I'm thinking of buying a mini PC for use as a Linux-powered home server (eg, for backup drives and additional storage, etc).

There seem to be quite a few of these on Amazon, does anyone have any experience with any of these, or others, or any general advice?

A left field suggestion - why not build and rent a Linux server on Amazon's EC2 cloud service? - see https://aws.amazon.com/

I mention this because I have spent a lot of the summer playing around with AWS, mostly out of curiosity as moving stuff (both hardware and software) onto cloud services is becoming a rapidly increasing trend in my industry which I've been needing to bone up on.

I've had a Windows 2016 server running on AWS for a few months to try out running backups of various applications I use at home and like you for additional storage. The first year's usage is free and subsequent running costs look to be very low.

You would of course need a good broadband connection and browser to access it from another PC which I assume you would have anyway. And of course you'll learn some great new skills if you work in IT!

wrt external drives... £4 gets you a 2 Tb usb stick from china.... (obvious caveats)

get 4 of those and you've a 8Gb mirrored RAID drive.. or 16Gb of striped disk. that's a lot of space domestically!

didds

didds wrote:wrt external drives... £4 gets you a 2 Tb usb stick from china.... (obvious caveats)

That looks like a scam.

didds wrote:get 4 of those and you've a 8Gb mirrored RAID drive.. or 16Gb of striped disk. that's a lot of space domestically!

I do not understand your arithmetic.

GeoffF100 wrote:

didds wrote:get 4 of those and you've a 8Gb mirrored RAID drive.. or 16Gb of striped disk. that's a lot of space domestically!

I do not understand your arithmetic.

to be fair I may have cocked up...
isn't it

2 x 4TB = 8 Tb stripe.
do that twice and mirror one stripe to the other.
qed - 8TB mirrored drive

4 x 4TB = 16Tb stripe.
qed - 16Tb drive

happy to be wrong - its a long time since I did any of this in anger

didds

didds wrote:

GeoffF100 wrote:

didds wrote:get 4 of those and you've a 8Gb mirrored RAID drive.. or 16Gb of striped disk. that's a lot of space domestically!

I do not understand your arithmetic.

to be fair I may have cocked up...
isn't it

2 x 4TB = 8 Tb stripe.
do that twice and mirror one stripe to the other.
qed - 8TB mirrored drive

4 x 4TB = 16Tb stripe.
qed - 16Tb drive

happy to be wrong - its a long time since I did any of this in anger

didds

That sounds ok, but your original drive was 2TB, not 4TB.

Scott.

didds wrote:wrt external drives... £4 gets you a 2 Tb usb stick from china.... (obvious caveats)

get 4 of those and you've a 8Gb mirrored RAID drive.. or 16Gb of striped disk. that's a lot of space domestically!

didds

I wonder what the lifespan would be, given that you are talking cheap disposable flash memory versus SSD, something designed with wear leveling, self checking and use of redundant storage*.

There are of course many trade offs. In principle your idea fits well with the concept of RAID (redundant array of inexpensive "disks").
Indeed a search shows that it has been done many times.
Personally I not only suspect that the offer you quote is a scam, but that doing the same with more trustworthy suppliers would simply lead to significant maintenence. I.E. having to replace one or more sticks every year. Not to mention the maintenence of the SD card containing the OS in this case.
Strangely search does not reveal any comments upon the lifespan achieved of such a USB stick RAID array..

It might also be quite slow when writing.

*Actually there are some cheap SSD's on the market that gain their cost advantage by reducing the amount of redundant storage.

swill453 wrote:That sounds ok, but your original drive was 2TB, not 4TB.

Scott.

Ah! Yes. Bugger! LOL.

Make that a 8Tb stripe or a 4TB mirror then! DOH!

didds

Urbandreamer wrote:
I wonder what the lifespan would be,

etc..

Yes. hence my inclusion of " (obvious caveats)". So I didnt have to repeat everything that people here in this group already seem to know.

didds

This thread reminded me of another backup option I was considering a while ago, Wasabi. Referred to in the video posted here by Infrasonic viewtopic.php?f=39&t=23189&p=304480#p304480

https://wasabi.com/

Haven’t taken the plunge yet.

Urbandreamer wrote:

didds wrote:wrt external drives... £4 gets you a 2 Tb usb stick from china.... (obvious caveats)

get 4 of those and you've a 8Gb mirrored RAID drive.. or 16Gb of striped disk. that's a lot of space domestically!

didds

I wonder what the lifespan would be, given that you are talking cheap disposable flash memory versus SSD, something designed with wear leveling, self checking and use of redundant storage*.

There are of course many trade offs. In principle your idea fits well with the concept of RAID (redundant array of inexpensive "disks").
Indeed a search shows that it has been done many times.
Personally I not only suspect that the offer you quote is a scam, but that doing the same with more trustworthy suppliers would simply lead to significant maintenence. I.E. having to replace one or more sticks every year. Not to mention the maintenence of the SD card containing the OS in this case.
Strangely search does not reveal any comments upon the lifespan achieved of such a USB stick RAID array..

It might also be quite slow when writing.

*Actually there are some cheap SSD's on the market that gain their cost advantage by reducing the amount of redundant storage.

Flash drives from good manufacturers do have wear levelling and redundant storage, but still are not as good as a good SSD. Poor write speed is not necessarily an issue with a Raspberry Pi NAS. The write speed in the Explaining Computers video was about 50 MB / sec. Gigabit Ethernet limits the speed to about 100 MB / sec anyway. That will easily be beaten by a good flash drive.

What we do not know here are the data volumes (total data to be stored and volume stored per day). We also do not know the number and nature of the computers will be accessing the shared storage. We have been told that there is no space for an additional small form factor PC. That may rule out multiple mains powered disk drives too. Perhaps the simplest solution is to put the shared storage on one of the existing computers, after adding another disk drive, perhaps. That solution is easy and might cost nothing:

https://www.howtogeek.com/176471/how-to ... and-linux/

Urbandreamer wrote:

didds wrote:wrt external drives... £4 gets you a 2 Tb usb stick from china.... (obvious caveats)

I wonder what the lifespan would be...

The main issue with these unbelievably cheap large capacity USB flash drives isn't lifespan but simply that they're a scam. The drive reports it is 2Tb and appears to be so but the actual storage is much lower and it just overwrites/throws away data when it's reached its actual storage capacity.

https://www.google.com/search?q=2tb+usb+flash+drives+scam

Just as a FYI as the thread has meandered around many of the server/NAS/VPS options available today...https://www.qnap.com/solution/qutscloud/en-us/

QuTScloud is a cloud-optimized version of the QNAP NAS operating system. Install QuTScloud on your preferred public clouds, start your cloud NAS, and benefit from flexible file management and collaboration that is unavailable from most cloud platforms. With QuTScloud you can immediately experience all the benefits of a QNAP NAS without any hardware requirements or complex maintenance of the cloud environment.

Cont.

Redmires wrote:

didds wrote:If its effectively a NAS you need then I'd consider a Raspberry Pi - there are plenty of turtorials/howtos online to make a NAS from one. Its as small as you'll likely ever find currently, and has a fully controllable OS (raspbian - version of linux). On top of the Pi itself you'll need some external hard drive provision which could be as simple as a usb drive(s).

Depending on how sexy you want to get you could incorporate RAID into that etc etc for data intergrity purposes

didds

That's my next project. Here's an excellent, no nonsense guide (none of the usual youtube whooping and hollering)

https://www.youtube.com/watch?v=bpvlEbdA6qI

I watched this yesterday and was surprised at the poor write performance of 49MB/s. I have a Synology NAS of much lower spec than a Pi4 and that manages about 70MB/s on large files. The hard disk I am using is old as well - a temporary replacement for the one that was in there that developed a fault. I might try it with an SSD later. I have a Pi4 I might test with an SSD as well.

If you have a large number of files to backup and/or what a quick backup, nothing beats a locally attached SSD, even through USB3. 1 Gbit Network performance will limit backups to around 100MB/s. You can at least double that with a USB3 attached SSD.

hiriskpaul wrote:I watched this yesterday and was surprised at the poor write performance of 49MB/s. I have a Synology NAS of much lower spec than a Pi4 and that manages about 70MB/s on large files. The hard disk I am using is old as well - a temporary replacement for the one that was in there that developed a fault. I might try it with an SSD later. I have a Pi4 I might test with an SSD as well.

If you have a large number of files to backup and/or what a quick backup, nothing beats a locally attached SSD, even through USB3. 1 Gbit Network performance will limit backups to around 100MB/s. You can at least double that with a USB3 attached SSD.

In the comments below the video it's pointed out that the HDD was NTFS formatted. If it was reformatted with EXT4 it would improve transfer speeds substantially.

Redmires wrote:

hiriskpaul wrote:I watched this yesterday and was surprised at the poor write performance of 49MB/s. I have a Synology NAS of much lower spec than a Pi4 and that manages about 70MB/s on large files. The hard disk I am using is old as well - a temporary replacement for the one that was in there that developed a fault. I might try it with an SSD later. I have a Pi4 I might test with an SSD as well.

If you have a large number of files to backup and/or what a quick backup, nothing beats a locally attached SSD, even through USB3. 1 Gbit Network performance will limit backups to around 100MB/s. You can at least double that with a USB3 attached SSD.

In the comments below the video it's pointed out that the HDD was NTFS formatted. If it was reformatted with EXT4 it would improve transfer speeds substantially.

Yes, I can definitely confirm that ext4 gives much better performance than ntfs. Backing up 3 large files, about 21Gb in total, to an 8 year old USB3 attached mechanical drive gave 32MB/s using ntfs and 85MB/s using ext4. CrystalDiskInfo gives 108MB sequential read, 98Mb write to the network share. So using a Pi4 as a NAS seems reasonable.

However, there are a few oddities. I cannot get my SSDs to work reliably with either ntfs or ext4.

Curiously as well, real world read testing seems much worse than writing. Reading back my 3 large files and writing to my local SSD gives only 35MB/s. Anyone know why that might be?

hiriskpaul wrote:However, there are a few oddities. I cannot get my SSDs to work reliably with either ntfs or ext4.

Curiously as well, real world read testing seems much worse than writing. Reading back my 3 large files and writing to my local SSD gives only 35MB/s. Anyone know why that might be?

SSD's rely on cache, the better (more expensive) the SSD the more cache it will have (generally SLC).
When you do large file transfers the cache gets saturated and the performance drops off a cliff, sometimes below that of a HDD!
No way out of it other than stumping up for decent branded high performance SSD's (Samsung et al).

In general it's best to have a good DRAM/SLC cache SSD as a boot drive, cheaper DRAMless/small cache as data drives are fine (unless you routinely handle large files).
Long life NAS specific SSD's are now available (WD red £,Seagate Ironwolf ££ and Synology £££ --very expensive with power failure caps et al like enterprise drives).

I think modern Synology NAS' are BTRFS by default these days rather then EXT*.