HYPTUSS Update versions 11.83 and b-w

[kiloran]

New versions are available at http://lemonfoolfinancialsoftware.weebl ... op-up.html

The changes:

1. Yahoo prices are now retrieved by the v10 API since v6 and v7 are currently no longer working. The version is selected in cell B12 of the Parameters sheet.
2. RNS data from Investegate was failing due to changes on the Investegate web page. This has now been fixed
3. Dividends from ADVFN were failing following changes to the ADVFN web page. This has now been fixed.

Hopefully, the new versions are bug-free.

The future of the Yahoo Finance API to retrieve share prices does seem questionable at the moment, so alternatives will be looked at.

--kiloran

[seagles]

After downloading the file I opened it as normal. however, I get a "Security Risk" warning from Microsoft a macro has been blocked because the source of the file is untrusted. So far have not been able to get around this using the help. Would/could this be part of the SSL change? Just a thought before I start to look deeper.
Microsoft 365. BTW.

Edited. Fixed, saved the file to my desktop then could change the properties on the "General" tab to tick the Unblock file. When I opened the file from the download that option was not available.

[kiloran]

After downloading the file I opened it as normal. however, I get a "Security Risk" warning from Microsoft a macro has been blocked because the source of the file is untrusted. So far have not been able to get around this using the help. Would/could this be part of the SSL change? Just a thought before I start to look deeper.
Microsoft 365. BTW.

I would not think it is SSL related, but try downloading from this test site and see if it makes any difference: http://tmffstest.weebly.com/
This Weebly site does not have SSL enabled

I suspect it's a false positive

--kiloran

[seagles]

We crossed posts. I fixed the security warning by following the microsoft help. I had to "save" the download first before I could tick the unblock in the general tab of the properties.

[kiloran]

We crossed posts. I fixed the security warning by following the microsoft help. I had to "save" the download first before I could tick the unblock in the general tab of the properties.

I checked the file at https://www.virustotal.com/ and 60 virus-checkers gave it a clean result (phew!)

--kiloran

[seagles]

We crossed posts. I fixed the security warning by following the microsoft help. I had to "save" the download first before I could tick the unblock in the general tab of the properties.

I checked the file at https://www.virustotal.com/ and 60 virus-checkers gave it a clean result (phew!)

--kiloran

Lucky for you. I have an .exe file that is over 20 years old that fails evertime. I know it needs to be worked on but cannot be bothered anymore. Ok 20 years ago we got away with "murder" in coding and things are a lot tighter now, which in the long term is "good".

[seagles]

Have updated 3 different portfolios and performed various tests and the changes look to be working fine. However, have noticed a problem with Sharecast data (and this seems to go back some time looking at snapshots from the portfolios), the P/E value for Latest yield is zero on all occasions, the forecast yield figures seem ok. I know it is not something related to your changes but just a heads up. It is not a field I use.
I know I say this a lot but many thanks for your and IAAGs work on this, has saved me considerable work.

[kiloran]

Have updated 3 different portfolios and performed various tests and the changes look to be working fine. However, have noticed a problem with Sharecast data (and this seems to go back some time looking at snapshots from the portfolios), the P/E value for Latest yield is zero on all occasions, the forecast yield figures seem ok. I know it is not something related to your changes but just a heads up. It is not a field I use.
I know I say this a lot but many thanks for your and IAAGs work on this, has saved me considerable work.

I think you mean Latest PE is OK but Forecast PE is zero.

That's a problem with the data on the Sharecast web page. See the Valuation tab under Key Financials at https://www.sharecast.com/equity/Vodafone_Group
I think all shares have had this problem for some time.

--kiloran

[TUK020]

Firstly, many thanks for the work you do to keep this tool running - really appreciated, very useful.

I have just downloaded the version bw for LibreOffice, and put my portfolio into it.
I am running LibreOffice Version: 6.4.7.2 on an up to date Linux Mint

When I run it to update prices, works fine,
Some minor gremlins to report:
Dividend/yield update
- it returns a funny on MCT - the text string "Jan, Apr, Jul, Oct".
- For VMID it gives a ????
All charts gives most charts
- but not for BT, SHEL, or Haleon. (this is not just the latest version, I think this has been going a while, but I didn't report it)

Thanks for getting this running again
tuk020

[staffordian]

I'm sure I've overlooked something obvious, but I can't see it...

I've downloaded the new version, copied and pasted stocks and share numbers, but when I click the 'Get Yahoo Prices' macro button, or any other button for that matter, the cursor changes to a four headed arrow and the button is selected rather than the macro running.

I'd be very grateful for any pointers on what I've done wrong.

TIA

Staffordian

EDIT - NOW SOLVED - I closed and re-opened the file and all seems fine. Still no idea why it did this though.

[seagles]

Have updated 3 different portfolios and performed various tests and the changes look to be working fine. However, have noticed a problem with Sharecast data (and this seems to go back some time looking at snapshots from the portfolios), the P/E value for Latest yield is zero on all occasions, the forecast yield figures seem ok. I know it is not something related to your changes but just a heads up. It is not a field I use.
I know I say this a lot but many thanks for your and IAAGs work on this, has saved me considerable work.

I think you mean Latest PE is OK but Forecast PE is zero.

That's a problem with the data on the Sharecast web page. See the Valuation tab under Key Financials at https://www.sharecast.com/equity/Vodafone_Group
I think all shares have had this problem for some time.

--kiloran

Whoops, of course. I was in a hurry to let you know tests worked, before my company arrived.

[kiloran]

Firstly, many thanks for the work you do to keep this tool running - really appreciated, very useful.

I have just downloaded the version bw for LibreOffice, and put my portfolio into it.
I am running LibreOffice Version: 6.4.7.2 on an up to date Linux Mint

When I run it to update prices, works fine,
Some minor gremlins to report:
Dividend/yield update
- it returns a funny on MCT - the text string "Jan, Apr, Jul, Oct".
- For VMID it gives a ????
All charts gives most charts
- but not for BT, SHEL, or Haleon. (this is not just the latest version, I think this has been going a while, but I didn't report it)

Thanks for getting this running again
tuk020

It looks like the AIC table has occasional hidden elements so the code slips up and reports the dividend dates rather than the yield. This is where the MCT data comes from: https://www.theaic.co.uk/companydata/mi ... come-trust
I'll have a look at the problem.
For VMID the ???? indicates that the yield cannot be found. I'm not aware of any website that allows easy reporting of yield for ETFs.
The charts issue is a problem with the Livecharts webpage. For some reason, it does not have charts for some common shares. It's been like that for a long time. You could try selecting the Trustnet source on the Parameters sheet, though that is not perfect. I can't find any other easily-reportable source for charts

--kiloran

[Itsallaguess]

Dividend/yield update

- For VMID it gives a ????

For VMID the ???? indicates that the yield cannot be found.

I'm not aware of any website that allows easy reporting of yield for ETFs.

I've been getting alternative yield figures via the Yahoo v10 API for a while now, which continue to corroborate quite well with the Sharecast figures and does include ETF yields...

The v10 API data from Yahoo for VMID gives a yield figure of 3.30% -



Source - https://query2.finance.yahoo.com/v10/finance/quoteSummary/vmid.l?modules=price%2CsummaryDetail


Which aligns with the yield figure provided by Vanguard themselves for VMID -



Source - https://www.vanguardinvestor.co.uk/investments/vanguard-ftse-250-ucits-etf-gbp-distributing/distributions

Cheers,

Itsallaguess

[kiloran]

For VMID the ???? indicates that the yield cannot be found.

I'm not aware of any website that allows easy reporting of yield for ETFs.

I've been getting alternative yield figures via the Yahoo v10 API for a while now, which continue to corroborate quite well with the Sharecast figures and does include ETF yields...

The v10 API data from Yahoo for VMID gives a yield figure of 3.30% -

Cheers,

Itsallaguess

Hey, that's great! Except it's Yahoo
I do wonder how much longer we can rely on it as a data source. But I'll add it to my To-Do list.

--kiloran

[Arborbridge]

I've been getting alternative yield figures via the Yahoo v10 API for a while now, which continue to corroborate quite well with the Sharecast figures and does include ETF yields...

The v10 API data from Yahoo for VMID gives a yield figure of 3.30% -

Cheers,

Itsallaguess

Hey, that's great! Except it's Yahoo
I do wonder how much longer we can rely on it as a data source. But I'll add it to my To-Do list.

--kiloran

Thank you for the effort you put in for the community.

For an ignoramus, could you explain the significance of API numbers 6-7-10 etc? I understand that the API is the way HYPTUSS interfaces to the site as required (I think!) but why the different versions? And do they just lie dormant, or does HYPTUSS pick the one which works at the time? And how can you invent API-11, as someone suggested, when you don't know what changes will happen at the website?

Thanks - and I expect I'm not the only one for whom this is a black art and a mystery.
Arb.

[Itsallaguess]

For an ignoramus, could you explain the significance of API numbers 6-7-10 etc?

I understand that the API is the way HYPTUSS interfaces to the site as required (I think!) but why the different versions? And do they just lie dormant, or does HYPTUSS pick the one which works at the time? And how can you invent API-11, as someone suggested, when you don't know what changes will happen at the website

API stands for 'Application Programming Interface', and these are usually software interfaces that serve a specific function, and expect some sort of 'input' and will often then provide some sort of 'output'.

For many years HYPTUSS has been happily using the v6 Yahoo share-price API to gain free, almost-live share-price information, and until recently it's been running quite happily since the inception of the tool.

One of the best things about the v6 and v7 API interfaces was that part of the API process was being able to 'pass' the v6 and v7 API a single LIST of EPIC's in one go via the API URL, and the v6 or v7 API would then almost-instantly 'pass-back' an array of share-prices based on that initial EPIC list.

The long-term reliability and rapid speed of those multi-EPIC v6 and v7 API processes has been great to have publicly available for free, but sadly, that time has now come to an end with Yahoo closing down the v6 and v7 API processes at their end.

The v10 and v11 API Yahoo processes are currently still working, but they're a little different to the v6 and v7 API processes in that they will only expect a 'single EPIC transaction' being passed via the defined URL at any one time, although a side-benefit of that API change is that the returned JSON data, which is the format of data that's returned by the v10 and v11 API's, is much more verbose on a per-share basis, and can contain a wide variety of EPIC-specific information, such as share-price, dividend-yield, and lots more, as you can see if you open the v10 API link below for VODAFONE in a new browser tab -

https://query2.finance.yahoo.com/v10/finance/quoteSummary/vod.l?modules=price%2CfinancialData%2CsummaryDetail

As you can see if you open the above URL, the speed of returned v10 and v11 API data in JSON form for Vodafone is still really quick, and so even though we can only now use these later v10 and v11 API processes using a single EPIC at a time, it's still actually a quite rapid return of price-data even when looking at a fairly chunky number of individual holdings.

The question now is how long these v10 and v11 API processes will continue to be viable, and it's just a case of crossing our fingers a little on that front at the moment, but given the v6 and v7 API's had previously been running really reliably for over 10 years now, I've got to say we've had a good run already...

Cheers,

Itsallaguess

[Arborbridge]

The question now is how long these v10 and v11 API processes will continue to be viable, and it's just a case of crossing our fingers a little on that front at the moment, but given the v6 and v7 API's had previously been running really reliably for over 10 years now, I've got to say we've had a good run already...

Cheers,

Itsallaguess

Wow, thank you for all the detail. As you say, we've had a good run and can only keep our fingers crossed. It'll be a fag going back to plugging in prices by hand, but if needs must ..... it would certainly be an incentive to cut down on the number of holdings!


Arb.

[kiloran]

The question now is how long these v10 and v11 API processes will continue to be viable, and it's just a case of crossing our fingers a little on that front at the moment, but given the v6 and v7 API's had previously been running really reliably for over 10 years now, I've got to say we've had a good run already...

Cheers,
Itsallaguess

Wow, thank you for all the detail. As you say, we've had a good run and can only keep our fingers crossed. It'll be a fag going back to plugging in prices by hand, but if needs must ..... it would certainly be an incentive to cut down on the number of holdings!

Arb.

I don't think we'll ever need to plug in prices by hand. We can always scrape the data from a web page, but this is slower and messier to program than using the Yahoo API, and also dependent upon a stable web page

And there are other stock price APIs, but they tend to require log-in accounts and may not cover so much UK data

--kiloran

[kyu66]

The long-term reliability and rapid speed of those multi-EPIC v6 and v7 API processes has been great to have publicly available for free, but sadly, that time has now come to an end with Yahoo closing down the v6 and v7 API processes at their end.
Itsallaguess

FYI, the V7 API version is still available, but you need to handle cookies and a crumb in the same session, easy in Python for a LibreOffice version. See the code below, 'requests' is used in place of 'urllib'.

Code: Select all

import requests
def GetYahooDataPy(symbols):
   try:
      header = {'Connection': 'keep-alive',
                'Expires': '-1',
                'Upgrade-Insecure-Requests': '1',
                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) \
                 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36'
               }
      with requests.Session() as s:
          # get cookies
          response = s.get('https://fc.yahoo.com',headers=header)
          # get crumb
          crumb = s.get('https://query2.finance.yahoo.com/v1/test/getcrumb',headers=header).text
          url_with_crumb = 'http://query1.finance.yahoo.com/v7/finance/quote?crumb=' + crumb
          url_with_crumb_and_symbols = url_with_crumb + '&symbols=' + symbols
          response = s.get(url_with_crumb_and_symbols,headers=header)
      return response.text

   except IOError:
     return 0
 

HTH
kyu66

[kiloran]

FYI, the V7 API version is still available, but you need to handle cookies and a crumb in the same session, easy in Python for a LibreOffice version. See the code below, 'requests' is used in place of 'urllib'.

Code: Select all

import requests
def GetYahooDataPy(symbols):
   try:
      header = {'Connection': 'keep-alive',
                'Expires': '-1',
                'Upgrade-Insecure-Requests': '1',
                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) \
                 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36'
               }
      with requests.Session() as s:
          # get cookies
          response = s.get('https://fc.yahoo.com',headers=header)
          # get crumb
          crumb = s.get('https://query2.finance.yahoo.com/v1/test/getcrumb',headers=header).text
          url_with_crumb = 'http://query1.finance.yahoo.com/v7/finance/quote?crumb=' + crumb
          url_with_crumb_and_symbols = url_with_crumb + '&symbols=' + symbols
          response = s.get(url_with_crumb_and_symbols,headers=header)
      return response.text

   except IOError:
     return 0
 

HTH
kyu66

Thanks, that's good to know but only useful for a stand-alone Python session. The requests module is not part of the standard Python application, and I don't believe it is possible to use add-in modules such as this in LibreOffice. It's got to be plain, native Python.

--kiloran