Shamba Upgrade by tmaabm · Pull Request #25 · shamba-admin/shamba

tmaabm · 2025-05-02T18:01:49Z

In this upgrade of the SHAMBA model, the following changes were made:

Upgrade Python from v2 to v3
Use a Python package manager: poetry
Remove the old GUI and replace it with a modern client-server application.
Enable interactive prompts when running the model using the command line.
Add type annotations to Python functions,
Vectorise most variables.
Convert all variable names to be snake cased instead of camel case.
Add model input validations using marshmallow.
Add flexibility by removing classes and prioritizing functions.
Break large functions/methods into smaller ones.
Fix bugs reported in the BGD report.
Parameterize some inputs, especially in the command line.
Improve the printing to stdout.
Add tests.
Updated the documentation.

N.B

The client-server application is a proof of concept and is not fully functional; e.g. some of the input cannot be defined in the UI.

- Commit example files when we should not really

hhughes92

(I have not thoroughly reviewed the test files yet)

I have put the GUI feedback on the Projects.civet file.

Perhaps worth discussing the comments: it will certainly be that some of them are for modellers to resolve and some are better handled by devs.

tmaabm · 2025-08-07T07:35:40Z

Pipfile

+[dev-packages]
+
+[requires]
+python_version = "3.13"


This is an artifact from an initial attempt to use pipenv. It is no longer needed. I'll remove it

hhughes92 · 2025-08-06T08:23:14Z

docs/user-guide.md

@@ -75,7 +75,7 @@ Then to enable a smooth debugging experience:
 4- choose the appropriate Python interpreter in vscode (bottom left in the status bar)
 You should choose something like ```Python 3.7.8 64-bit ('myenv':conda)```


Need to make sure python versions are consistent, where stated.

I've made corrections to this specific issue but I wonder if this README is still relevant. Should we just remove it?

Yes, I think so. Any information that's still important can be moved into one of the remaining three READMEs, which should cover everything.

shamba/README.md: what is SHAMBA, links to model description, explains what's in the other READMEs

shamba/shamba/README.md: explains how to set up the environment, run the command_line script, run the UI version

shamba/shamba/client/README.md: explains how to generate the static files for the UI

[DOCUMENTATION]

shamba/README.md

+
+#### Run the Server
+
+From the `./server` directory, run the server in _development_ mode: `poetry run fastapi dev main.py`


hhughes92 · 2025-08-06T08:33:04Z

README.md

@@ -0,0 +1,12 @@
+# Small-Holder Agriculture Mitigation Benefit Assessment (SHAMBA) project. version 2.0


Version number tbc

[DOCUMENTATION]

hhughes92 · 2025-08-06T08:43:35Z

versions.md

We should probably clean this up to reflect what has been done in this latest update.

This PR's description could be a template but I'm not sure what exactly we want to include/exclude?

We can do this at the end after modelling tasks are resolved, too.

[DOCUMENTATION]

shamba/tests/test_fertiliser_emissions.py

+        freq=int(csv_input_data["base_sf_int"]),
+        qty=float(csv_input_data["base_sf_qty"]),


shamba/tests/test_fertiliser_emissions.py

+        freq=int(csv_input_data["proj_sf_int"]),
+        qty=float(csv_input_data["proj_sf_qty"]),


shamba/tests/test_litter_emissions.py

+        litterFreq=int(csv_input_data["base_lit_int"]),
+        litterQty=float(csv_input_data["base_lit_qty"]),


shamba/tests/test_litter_emissions.py

+        litterFreq=int(csv_input_data["proj_lit_int"]),
+        litterQty=float(csv_input_data["proj_lit_qty"]),
+        no_of_years=N_YEARS,


shamba/tests/test_tree_emissions.py

+from model.common.calculate_emissions import get_int
+
+
+def test_crop_model():


hhughes92

Missed climate.py in previous review

hhughes92 · 2025-08-06T11:22:30Z

shamba/model/climate.py

+from model.common.data_sources.climate import get_climate_data
+
+
+def validate_list_length(lst):


Suggest this should either be called validated_monthly_list_length and assume length should be 12 or have the intended length as a separate arg? could be used to make sure yearly lists are the length of NUM YEARS

hhughes92 · 2025-08-06T11:23:16Z

shamba/model/climate.py

+    )
+
+    # Account for scaling factor in CRU-TS dataset
+    climate_data *= 0.1


[Modellers] Does this apply to every indicator? We are also not using CRU-TS from the new API

hhughes92 · 2025-08-06T11:24:48Z

shamba/model/climate.py

+    return schema.load(params)  # type: ignore
+
+
+def from_csv(


This isn't called anywhere? Care needed on is_evaporation.

hhughes92 · 2025-08-06T11:24:59Z

shamba/model/climate.py

+    climate_data *= 0.1
+
+    # Get the number of days in every month
+    days_in_months = np.array([calendar.monthrange(2000, i)[1] for i in range(1, 13)])


[Modellers] Is it correct to use a leap year here?

hhughes92 · 2025-08-15T10:23:49Z

Update 15/08/25:

resolved comments hidden
🚀reaction = comment resolved on HH local branch (waiting for repo permissions to be granted to push)

Consolidate some logic

tmaabm · 2025-08-20T06:41:31Z

@hhughes92 I've merged the other branch to this one. Hopefully there won't be any conflicts with you branch. I recommend you do git rebase as soon as you can.

tmaabm added 30 commits February 3, 2025 15:16

ignore .DS_Store files

38e2f47

chore: populate Dockerfile

e78263c

chore: use 2to3 in with shamba_cl_.py

59775af

chore: upgrade other files

869cc9b

chore: more upgrades

30d537a

chore: Fix runtime issues

392d907

- Commit example files when we should not really

chore: re-organise and rename folders and files

ca51d82

chore: simplify file names

9a049ee

chore: make input entry interactive

e6e6310

chore: remove duplicate modules

6be3684

chore: migrate from pipenv to poetry

3294eb2

chore: re-add pyproject.toml

7fda396

chore: add poetry.lock

14a2979

chore: update the Climate model and format files

47ef601

chore: add model validations to crop_model and crop_params

aa50671

chore: remove unused classes

3153344

chore: remove unused __init__

ea7b413

chore: simplify emit module

552b811

chore: remove Emissions class and fix plot

46be08d

chore: add validations to litter

63c4ed9

chore: add validations to soil params

fa482b8

chore: clean up soil_params module

141d117

chore: add validations to tree_params

e65a917

chore: add model validations to tree_growth

3793a15

chore: add model validations to tree_model

11e0792

chore: update soil models

6db4b1f

chore: soil model runs after validations

5b8078e

chore: add test infrastructure

6d5fe92

chore: adding first test for crop model

72ca269

chore: parameterise N_YEAR

235c60d

tmaabm added 7 commits April 29, 2025 09:43

chore: better function name

1f0bf4c

chore: small improvements

2c262db

chore: more camel case to snake case

41d9d7f

chore: finish off camelCase to snake_case

f511986

chore: update READMEs

6bdf721

chore: re-add root README

0158fab

chore: fix getting results in UI

c8a0bb6

tmaabm force-pushed the cirevo/initial-improvements branch from 91d0e2d to c8a0bb6 Compare June 4, 2025 13:51

tmaabm added 4 commits July 30, 2025 15:27

chore: specify gdal version

102733d

chore: update README

459a037

chore: update READMEs

0f9d894

chore: ignore all files in static but the index.html file

c53a198

hhughes92 reviewed Aug 6, 2025

View reviewed changes

tmaabm added 7 commits August 6, 2025 18:23

chore: dockerisation phase 2

2c00a12

chore: fully dockerised

ed4eb12

chore(code-review): address some comments

70a4152

chore: address more comments

ce1cd1f

chore: consolidate from_csv

14be601

chore: remove print statement

03bc14e

chore: include tests in the docker image

0a2f83f

tmaabm added 7 commits August 16, 2025 16:09

chore: abstracting soil model

183e5c9

chore: abstract soil model

0a87dd6

chore: conditionally print to stdout

6a2bc9b

chore: parameterise soil model in terminal

edbae4c

chore: update README with docker instructions

aa0fe34

chore: rename roth-specific variable names and keys

59959b1

Merge pull request #26 from rstedham/consolidate-some-logic

c04c413

Consolidate some logic

		@@ -75,7 +75,7 @@ Then to enable a smooth debugging experience:
		4- choose the appropriate Python interpreter in vscode (bottom left in the status bar)
		You should choose something like ```Python 3.7.8 64-bit ('myenv':conda)```


		#### Run the Server

		From the `./server` directory, run the server in _development_ mode: `poetry run fastapi dev main.py`

		@@ -0,0 +1,12 @@
		# Small-Holder Agriculture Mitigation Benefit Assessment (SHAMBA) project. version 2.0

		freq=int(csv_input_data["base_sf_int"]),
		qty=float(csv_input_data["base_sf_qty"]),

		freq=int(csv_input_data["proj_sf_int"]),
		qty=float(csv_input_data["proj_sf_qty"]),

		litterFreq=int(csv_input_data["base_lit_int"]),
		litterQty=float(csv_input_data["base_lit_qty"]),

		from model.common.calculate_emissions import get_int


		def test_crop_model():

		from model.common.data_sources.climate import get_climate_data


		def validate_list_length(lst):

Conversation

tmaabm commented May 2, 2025

Uh oh!

hhughes92 left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmaabm Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

hhughes92 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hhughes92 commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tmaabm commented Aug 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tmaabm Aug 7, 2025 •

edited

Loading

hhughes92 commented Aug 15, 2025 •

edited

Loading