💹 justETF Scraping

Scrape the justETF.

🛠️ Installation

To use justETF scraping package in your project, install the actual version from GitHub:

pip install git+https://github.com/druzsan/justetf-scraping.git

If you are going to play notebooks through, use the following installation:

pip install justetf-scraping[all]@git+https://github.com/druzsan/justetf-scraping.git

🚀 Usage

📋 Scrape the justETF Screener

Load overviews for all available (over 3400 at the moment) ETFs. By default, it loads long-only, active, short & leveraged ETFs (3 requests), but without further enrichments:

import justetf_scraping

df = justetf_scraping.load_overview()
df

	wkn	ticker	valor	name	inception_date	age_in_days	age_in_years	strategy	domicile_country	currency	...	last_year_volatility	last_three_years_volatility	last_five_years_volatility	last_year_return_per_risk	last_three_years_return_per_risk	last_five_years_return_per_risk	max_drawdown	last_year_max_drawdown	last_three_years_max_drawdown	last_five_years_max_drawdown
isin
IE00B5BMR087	A0YEDG	SXR8	10737041	iShares Core S&P 500 UCITS ETF USD (Acc)	2010-05-19	5738	15.720548	Long-only	Ireland	USD	...	18.97	15.85	17.58	0.04	1.12	0.86	-33.71	-22.60	-22.60	-22.60
IE00B4L5Y983	A0RPWH	EUNL	10608388	iShares Core MSCI World UCITS ETF USD (Acc)	2009-09-25	5974	16.367123	Long-only	Ireland	USD	...	15.38	13.20	14.73	0.26	1.23	0.91	-33.91	-20.45	-20.45	-20.45
IE00B3XXRP09	A1JX53	VUSA	18575508	Vanguard S&P 500 UCITS ETF (USD) Distributing	2012-05-22	5004	13.709589	Long-only	Ireland	USD	...	19.69	16.20	17.81	0.04	1.10	0.85	-33.70	-23.74	-23.74	-23.74
IE00B4ND3602	A1KWPQ	PPFB	12881542	iShares Physical Gold ETC	2011-04-08	5414	14.832877	Long-only	Ireland	USD	...	19.76	16.04	15.57	2.90	2.09	1.43	-37.20	-7.86	-7.86	-11.73
IE00B3YCGJ38	A1CYW7	P500	11358996	Invesco S&P 500 UCITS ETF Acc	2010-05-20	5737	15.717808	Long-only	Ireland	USD	...	19.17	15.99	17.67	0.05	1.13	0.87	-33.62	-22.61	-22.61	-22.61
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
XS2427474023	A3GWVN	O2IG	NaN	WisdomTree STOXX Europe Oil & Gas 2x Daily Short	2022-03-02	1433	3.926027	Short & Leveraged	Ireland	EUR	...	38.06	36.21	NaN	-1.21	-0.69	NaN	-77.65	-59.72	-62.49	NaN
XS2437455608	A3GXB6	STR2	NaN	WisdomTree STOXX Europe Travel & Leisure 2x Da...	2022-03-02	1433	3.926027	Short & Leveraged	Ireland	EUR	...	39.58	36.97	NaN	-0.25	-0.49	NaN	-75.87	-50.62	-56.76	NaN
XS3037640110	NaN	SBA3	NaN	Leverage Shares -3x Short Alibaba (BABA) ETP S...	2022-06-07	1336	3.660274	Short & Leveraged	Ireland	USD	...	156.89	129.87	NaN	-0.61	-0.61	NaN	-99.93	-96.88	-99.66	NaN
XS2531767767	NaN	5SIT	NaN	GraniteShares 5x Short MIB Daily ETF	2023-06-09	969	2.654795	Short & Leveraged	Ireland	EUR	...	88.05	NaN	NaN	-0.93	NaN	NaN	-97.70	-87.80	NaN	NaN
XS2842095759	NaN	3SBB	NaN	GraniteShares 3x Short Alibaba Daily ETP	2022-02-02	1461	4.002740	Short & Leveraged	Ireland	USD	...	150.08	5696.15	NaN	-0.64	-0.01	NaN	-99.99	-96.79	-99.99	NaN

4068 rows × 42 columns

To reduce the number of requests, one particular ETF type can be loaded, e.g. long-only ETFs:

df = justetf_scraping.load_overview(strategy="epg-longOnly")

To further enrich the data with additional information (asset class, region, exchanges and instrument, multiple requests for each combination required):

df = justetf_scraping.load_overview(enrich=True)

Load ETFs for a single index, e.g. MSCI World:

df = justetf_scraping.load_overview(strategy="epg-longOnly", index="MSCI World")

📈 Scrape ETF Chart Data from justETF (e.g.)

Load the whole history of a chosen ETF by its ISIN:

df = justetf_scraping.load_chart("IE00B0M62Q58")
df

	quote	relative	dividends	cumulative_dividends	quote_with_dividends	relative_with_dividends	reinvested_dividends	quote_with_reinvested_dividends	relative_with_reinvested_dividends
date
2005-10-28	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-29	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-30	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-31	20.99	1.893204	0.0	0.00	20.99	1.893204	0.000000	20.990000	1.893204
2005-11-01	21.03	2.087379	0.0	0.00	21.03	2.087379	0.000000	21.030000	2.087379
...	...	...	...	...	...	...	...	...	...
2026-01-26	81.57	295.970874	0.0	10.98	92.55	349.271845	27.130307	108.700307	427.671393
2026-01-27	81.40	295.145631	0.0	10.98	92.38	348.446602	27.073765	108.473765	426.571674
2026-01-28	80.95	292.961165	0.0	10.98	91.93	346.262136	26.924094	107.874094	423.660651
2026-01-29	80.88	292.621359	0.0	10.98	91.86	345.922330	26.900812	107.780812	423.207825
2026-01-30	80.88	292.621359	0.0	10.98	91.86	345.922330	26.900812	107.780812	423.207825

7400 rows × 9 columns

For accumulating ETFs, all dividends columns are zeros and all columns with dividends are equal to the ones without.

If you want to include quote from not yet closed day (today):

df = justetf_scraping.load_chart("IE00B0M62Q58", unclosed=True)

Compare multiple charts. It will take the shortest time period and compare gain as percentage including payed out but not reinvested dividends:

df = justetf_scraping.compare_charts(
    {
        "IE00B0M62Q58": justetf_scraping.load_chart("IE00B0M62Q58"),
        "IE00B0M63177": justetf_scraping.load_chart("IE00B0M63177"),
    },
)
df

	IE00B0M62Q58	IE00B0M63177
date
2005-11-18	0.000000	0.000000
2005-11-19	0.000000	0.000000
2005-11-20	0.000000	0.000000
2005-11-21	-0.539811	-0.887436
2005-11-22	0.629780	-0.934143
...	...	...
2026-01-26	316.329285	190.191499
2026-01-27	315.564552	191.359178
2026-01-28	313.540261	194.722092
2026-01-29	313.225371	194.581971
2026-01-30	313.225371	191.452592

7379 rows × 2 columns

💗 Scrape Live Quote

Currently, only Gettex exchange and Euro currency are supported by live quote.

Load the last quote of a chosen ETF by its ISIN:

quote = justetf_scraping.load_live_quote("IE00B0M62Q58")
quote

Quote(isin='IE00B0M62Q58', timestamp=datetime.datetime(2026, 2, 2, 15, 43, 30, 201000, tzinfo=datetime.timezone.utc), exchange='gettex', currency='EUR', trend=None, ask=82.0, bid=81.98, mid=81.99, last=81.19, spread=0.02, spread_relative=0.0002, spread_percentage=0.02, day_to_day=0.8, day_to_day_relative=0.0099, day_to_day_percentage=0.99)

Subscribe to live quote of a chosen ETF by its ISIN:

for quote in justetf_scraping.iterate_live_quote("IE00B0M62Q58"):
    print(quote)  # Add your processing here

Quote(isin='IE00B0M62Q58', timestamp=datetime.datetime(2026, 2, 2, 15, 43, 30, 201000, tzinfo=datetime.timezone.utc), exchange='gettex', currency='EUR', trend=None, ask=82.0, bid=81.98, mid=81.99, last=81.19, spread=0.02, spread_relative=0.0002, spread_percentage=0.02, day_to_day=0.8, day_to_day_relative=0.0099, day_to_day_percentage=0.99)
Quote(isin='IE00B0M62Q58', timestamp=datetime.datetime(2026, 2, 2, 15, 43, 32, 270000, tzinfo=datetime.timezone.utc), exchange='gettex', currency='EUR', trend=None, ask=82.0, bid=81.98, mid=81.99, last=81.19, spread=0.02, spread_relative=0.0002, spread_percentage=0.02, day_to_day=0.8, day_to_day_relative=0.0099, day_to_day_percentage=0.99)
Quote(isin='IE00B0M62Q58', timestamp=datetime.datetime(2026, 2, 2, 15, 43, 34, 656000, tzinfo=datetime.timezone.utc), exchange='gettex', currency='EUR', trend=None, ask=82.0, bid=81.97, mid=81.99, last=81.19, spread=0.03, spread_relative=0.0004, spread_percentage=0.04, day_to_day=0.8, day_to_day_relative=0.0099, day_to_day_percentage=0.99)
...

So you can react to quote change. The update frequency cannot be controlled. No updates besides the initial quote will be received outside of trade hours.

🔍 Scrape ETF Profile Data

Get comprehensive ETF profile data including description, holdings allocation by country and sector, and real-time quotes from gettex.

Get complete ETF overview with get_etf_overview

overview = justetf_scraping.get_etf_overview("IE00B3RBWM25")

# Access basic info
print(f"Name: {overview['name']}")
print(f"TER: {overview['ter']}%")
print(f"Fund Size: EUR {overview['fund_size_eur']}m")
print(f"Description: {overview['description']}")

# Access country allocation (full list, not truncated)
for country in overview['countries']:
    print(f"  {country['name']}: {country['percentage']}%")

# Access sector allocation (full list)
for sector in overview['sectors']:
    print(f"  {sector['name']}: {sector['percentage']}%")

# Access top 10 holdings with their ISINs
for holding in overview['top_holdings']:
    print(f"  {holding['name']} ({holding['isin']}): {holding['percentage']}%")

# Access real-time gettex quote
quote = overview['gettex']
print(f"Bid: {quote['bid']} {quote['currency']}")
print(f"Ask: {quote['ask']} {quote['currency']}")
print(f"Day Change: {quote['day_change_percent']}%")

The get_etf_overview function returns a dictionary with:

Field	Type	Description
isin	str	ISIN code
name	str	ETF name
description	str	Short description
index	str	Tracked index
ter	float	Total Expense Ratio (e.g., 0.19)
fund_size_eur	float	Fund size in EUR millions
replication	str	Replication method
fund_currency	str	Fund currency
distribution_policy	str	Distributing/Accumulating
inception_date	str	Launch date
fund_domicile	str	Country of domicile
countries	list	Full country allocation
sectors	list	Full sector allocation
top_holdings	list	Top 10 holdings with ISINs
gettex	dict	Real-time quote data

For a complete example, see test_scrape.py

For further exploration examples, see Jupyter Notebooks

⚒️ Development Setup

Prerequisites:

• Python 3.10 or later
• uv for dependency management
• make (optional) for shortcut commands
• direnv (optional) to automate enviroment setup
• asdf (optional) to manage direnv, Python and uv versions. For correct direnv setup, use pre 0.16 version.

Setup project using asdf, direnv and make

asdf add plugin python
asdf add plugin uv
asdf add plugin direnv
asdf direnv setup --shell bash --version latest
# Navigate to project folder
asdf install
direnv allow
make init
# To see more make targets, run:
make help

Setup project using pip (Python should be preinstalled):

pip install uv
# Navigate to project folder
uv sync --all-extras
uv run pre-commit install
# To run any environment-related commands, use `uv run ...`, e.g.:
uv run notebook

Optionally, use make targets for predefined commands, e.g. make sync to install all dependencies. Run make help to see all commands.

Similar Projects

• https://github.com/AshNL/justETF-overview-scraper
• https://github.com/SimonMandlik/etf_filter

Thanks

This project was inspired by this Stack Overflow question.