Group 7: Energy Efficiency of Quantized vs Full-Precision LLM Inference#169
Open
AmyTUD wants to merge 1 commit intoluiscruz:mainfrom
Open
Group 7: Energy Efficiency of Quantized vs Full-Precision LLM Inference#169AmyTUD wants to merge 1 commit intoluiscruz:mainfrom
AmyTUD wants to merge 1 commit intoluiscruz:mainfrom
Conversation
Collaborator
|
Dear Group 7, |
Author
Is it okay like this? |
Collaborator
|
yes @AmyTUD thank you! 🙏🏼 |
lacinoire
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use only this template to open a pull request for project 1: Measuring Energy Consumption.
Use the title below for your pull request title.
Group 7: Energy Efficiency of Quantized vs Full-Precision LLM Inference
Make sure to fill out the information under each of the headers.
Open the pull request from a repository + branch everyone in your group has access to, and use that branch to contiuously update your work throughout the project weeks.
Group number on Brightspace:
7
Group members (only names, leave out student numbers):
(You will have been assigned a group of 4 members on Brightspace)
Ceylin Ece, Georgios Markozanis, Kunal Narwani, Amy van der Meijden
Agreed communication channel within the group:
WhatsApp
Did you manage to contact all group members?:
Yes
Your topic idea for Project 1:
(We will review the topics and comment on the PRs whether the projects are appropriate or need adjustment. Don't hesitate to come up to us after the lectures / in the labs to discuss project ideas)
Energy Efficiency of Quantized vs Full-Precision LLM Inference
Large Language Models are energy-intensive, but quantization techniques promise to reduce their computational demands. This project compares the energy consumption of running identical prompts through small LLMs (Llama 3.2 1-3B) in both full-precision (fp16) and 4-bit quantized (GGUF) formats. We will measure energy consumption, throughput, and quality trade-offs to provide empirical data for sustainable AI deployment decisions.
Filename of your Project 1 blog post in
p1_measuring_software/(contributed in this pull request):(Fill out the yaml header fitting to your group)
g7_llm_quantization.md
Did you succeed to build the website locally and look at your blogpost?
yes