Donate to Remove ads

Got a credit card? use our Credit Card & Finance Calculators

Thanks to johnstevens77,Bhoddhisatva,scotia,Anonymous,Cornytiv34, for Donating to support the site

Archived Fool posts

For discussion of the practicalities of setting up and operating income-portfolios which follow the HYP Group Guidelines. READ Guidelines before posting
Forum rules
Tight HYP discussions only please - OT please discuss in strategies
Breelander
Lemon Quarter
Posts: 4178
Joined: November 4th, 2016, 9:42 pm
Has thanked: 999 times
Been thanked: 1855 times

Archived Fool posts

#544

Postby Breelander » November 4th, 2016, 11:07 pm

Many of us would like to refer back our old HYPP posts, or reference significant posts of others.

In the highly unlikely event that TMF ever let us have a copy of their database (all bulletin boards like this and TMF are databases and the 'posts' we read are actually database queries) the task of translating the record structure from one BB format to another is daunting. That's for modern well-structured BB systems for which the record format is known and documented - the TMF one was a bespoke in-house design unlike any other (by necessity, there were no full-feature BBs at the time). The world moved on, and the TMF design didn't. I think we have to accept we will never be able to 'extract' the old posts.

Fortunately we don't have to. The WayBack Machine has already archived many of them. Hopefully the old boards will just go 'read-only', but should they disappear for good we can still find a copy to link to. For example, this is a post I made on unitisation, which has links to the posts I learned it from. All are in the WayBack Machine for posterity....

https://web.archive.org/web/20120314180716/http://boards.fool.co.uk/brees-hypish-units-12469453.aspx
(apparently we can't post links, so you'll have to copy and paste this into your browser's address bar)

....well, they are now :) My post was already there, but the two I linked to weren't. Trying to follow the links, the Wayback Machine said...
This page is available on the web!
Help make the Wayback Machine more complete!

Save this url in the Wayback Machine

...so I did (save it, that is). The pages I asked to be saved contained valuable contributions from TJH and Gengulphus.

The limitation of the Internet Archive is that you can't easily search for a particular post, so get working now while we can still search the Fool. Find those posts you regard as significant and see if https://archive.org/web/ has already archived them. If not, ask it to!

Bree.

Breelander
Lemon Quarter
Posts: 4178
Joined: November 4th, 2016, 9:42 pm
Has thanked: 999 times
Been thanked: 1855 times

Re: Archived Fool posts

#554

Postby Breelander » November 4th, 2016, 11:18 pm

This is on such post I regard as worthy of archiving. It's another unitisation one from 2007 featuring both Gengulphus and TJH.

https://web.archive.org/web/20161104231238/http://boards.fool.co.uk/unitisation-10581966.aspx?sort=whole

spiderbill
Lemon Slice
Posts: 543
Joined: November 4th, 2016, 9:12 am
Has thanked: 155 times
Been thanked: 182 times

Re: Archived Fool posts

#821

Postby spiderbill » November 5th, 2016, 4:03 pm

Good point Bree, and worth bringing to the attention of anyone like Gengulphus who has posted valuable material and wants to retain access without downloading locally.

cheers

Spiderbill

Gengulphus
Lemon Quarter
Posts: 4255
Joined: November 4th, 2016, 1:17 am
Been thanked: 2628 times

Re: Archived Fool posts

#836

Postby Gengulphus » November 5th, 2016, 4:24 pm

spiderbill wrote:Good point Bree, and worth bringing to the attention of anyone like Gengulphus who has posted valuable material and wants to retain access without downloading locally.


I want to retain access to valuable material with downloading locally! That doesn't mean that I want my local copy to be the only way of accessing it, or even the main one, but I want to have it as a backup in case the non-local copy vanishes... One of the lessons of what's happening to TMF for me is that that can happen rather unexpectedly - though fortunately the vanishing process is fairly slow on this occasion!

That doesn't mean that I want to keep local copies of everything, by the way - a good thing too given that I have 20,000+ posts on TMF! The amount of valuable material is fortunately much lower.

Gengulphus

tjh290633
Lemon Half
Posts: 8208
Joined: November 4th, 2016, 11:20 am
Has thanked: 913 times
Been thanked: 4096 times

Re: Archived Fool posts

#854

Postby tjh290633 » November 5th, 2016, 5:25 pm

Breelander wrote:Save this url in the Wayback Machine

...so I did (save it, that is). The pages I asked to be saved contained valuable contributions from TJH and Gengulphus.

The limitation of the Internet Archive is that you can't easily search for a particular post, so get working now while we can still search the Fool. Find those posts you regard as significant and see if https://archive.org/web/ has already archived them. If not, ask it to!

Bree.


Thanks for doing that, Bree. Most of the data that I have posted has a URL noted in my files, but am not sure about finding them on Wayback.

TJH

Breelander
Lemon Quarter
Posts: 4178
Joined: November 4th, 2016, 9:42 pm
Has thanked: 999 times
Been thanked: 1855 times

Re: Archived Fool posts

#856

Postby Breelander » November 5th, 2016, 5:41 pm

tjh290633 wrote: Most of the data that I have posted has a URL noted in my files, but am not sure about finding them on Wayback.

TJH

Go to the Wayback home page https://archive.org/web/ Type or paste the address of the page you want in the address box at the top, the one between the 'Wayback' logo and the 'BROWSE HISTORY' button. If it already has that page it will find it for you. If it doesn't it will say so, then you can click the Save this url in the Wayback Machine link and it will archive it for you.

tjh290633
Lemon Half
Posts: 8208
Joined: November 4th, 2016, 11:20 am
Has thanked: 913 times
Been thanked: 4096 times

Re: Archived Fool posts

#859

Postby tjh290633 » November 5th, 2016, 5:49 pm

Thanks, Bree. I tried one which I found was already there. I must have a serious look at some of my links.

TJH

Deev8
Lemon Pip
Posts: 62
Joined: November 7th, 2016, 11:11 pm
Has thanked: 3 times
Been thanked: 6 times

Re: Archived Fool posts

#1982

Postby Deev8 » November 7th, 2016, 11:33 pm

Many of us would like to refer back our old HYPP posts, or reference significant posts of others ... The limitation of the Internet Archive is that you can't easily search for a particular post, so get working now while we can still search the Fool. Find those posts you regard as significant and see if https://archive.org/web/ has already archived them. If not, ask it to!


Excellent suggestion - I tried it with an old (almost ancient) post of mine that I occasionally provide as an answer to a basic question - https://web.archive.org/web/20110711132053/http://boards.fool.co.uk/how-to-calculate-your-hyps-performance-10784327.aspx

Dave

Infrasonic
Lemon Quarter
Posts: 4479
Joined: November 4th, 2016, 2:25 pm
Has thanked: 644 times
Been thanked: 1260 times

Re: Archived Fool posts

#2333

Postby Infrasonic » November 8th, 2016, 5:39 pm

For those that want to save without filling up their hard drives you could always use your free cloud storage allocations via browser extensions like Evernote, OneNote (OneDrive), Google Keep, or Google Drive 'clippers'.

MyNameIsUrl
Lemon Slice
Posts: 474
Joined: November 4th, 2016, 1:56 pm
Has thanked: 1304 times
Been thanked: 107 times

Re: Archived Fool posts

#3453

Postby MyNameIsUrl » November 10th, 2016, 7:55 pm

Infrasonic wrote:For those that want to save without filling up their hard drives you could always use your free cloud storage allocations via browser extensions like Evernote, OneNote (OneDrive), Google Keep, or Google Drive 'clippers'.


As an experiment to see how much file space previous posts take up, I just did a copy-and-paste of a 100-post thread into Notepad. Obviously the formatting was a mess, bu it gave me a file of 111 KB. On that basis my guess is that the whole TMF archive (20 million posts?) would fit on a USB pen drive. If there were a way of downloading a whole board in a searchable way we could all keep a personal copy of the HYP board.

ManInTheStreet
Posts: 35
Joined: November 4th, 2016, 3:29 pm
Has thanked: 7 times
Been thanked: 12 times

Re: Archived Fool posts

#3480

Postby ManInTheStreet » November 10th, 2016, 8:39 pm

There's no quick way. And as you say there is a lot of data, but not so much that it's not possible.

This bash script works to get all the messages up to number 13456772. Change the value of id in the embedded awk script to change the latest message. Change the zero in the while command to set the earliest.

Note, this is for Linux. I don't use Windows because I don't know a way to do this sort of thing on Windows this efficiently. Note that wget does all the grunt work.

Code: Select all

#!/bin/bash
# Each message has an ID. You can get a message with a given ID
# using a URL such as:
#
#      http://boards.fool.co.uk/Message.aspx?Mid=13456772
#
# This will return message number 13456772 (this is from micrographia with
# subject: Re: Closure of boards). It will stop when you get to message 1. This
# will take a very long time (maybe a month?).
#-------------------------------------------------------------------------------

awk 'BEGIN {
   id  = 13456772
   url = "http://boards.fool.co.uk/Message.aspx?Mid="
   while(id > 0) {
      cmd = "wget -Omsg" id ".html " url id " 2>/dev/null"
      print cmd
      system(cmd)
      id--
   }
}
' /dev/null



Return to “HYP Practical (See Group Guidelines)”

Who is online

Users browsing this forum: Google Adsense [Bot] and 13 guests