Resources related to the research work "LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning" 1. Dependency The dependencies are in `requirements.txt`, use`pip install -r reuirements.txt ` to install required libraries. 2. Brief Introduction LLaMA-Reviewer, an innovative framework that leverages the capabilities of LLaMA, a popular LLM, in the realm of code review. Mindful of resource constraints, this framework employs parameter-efficient fine-tuning (PEFT) methods, delivering high performance while using less than 1% of trainable parameters. An extensive evaluation of LLaMA-Reviewer is conducted on two diverse, publicly available datasets. Notably, even with the smallest LLaMA base model consisting of 6.7B parameters and a limited number of tuning epochs, LLaMA-Reviewer equals the performance of existing code-review-focused models. 3. Project Structure For baselines that don't have immediate results, testing code is provided. The necessary datasets and models can be acquired from the respective repositories given. In the case of LoRA, it's necessary to add the `xturing-base-model` to the weights folders. For prefix-tuning, the 'lit-llama-base-model' should be added to this location: `prefix(lit-llama)/lit-llama-main/checkpoints/lit-llama/7B`. The data has been omitted, but can be generated using the provided code and the original data from the repositories of two papers mentioned in the study. The results and outputs are preserved in the provided 7-zip files. The base models in the deposit are compressed by volumes, and they can also be obtained as per the guidelines of the corresponding framework. . ├── baselines # Baselines that don't have immediate results │ ├── AUGER │ ├── CommentFinder │ └── Tufano ├── LoRA(xturing) # Low-rank adaptation experiments, with LoRA rank of 8 or 16 │ ├── r=16 │ └── r=8 ├── prefix(lit-llama) # Prefix-tuning expereiments │ └── lit-llama-main ├── lit-llama-base-model # Base model for prefix-tuning │ └── lit-llama.pth ├── xturing-base-model # Base model for LoRA │ └── pytorch_model.bin └── requirements.txt # Requirements Note: in the `r=8` folder, "code alpaca" means "PL+NL" data, while "code alpaca only code" means "only PL" Note: in the `r=16` folder, "code alpaca" means "only PL" Note(new): Now the PEFT and the finetuning process have been well supported by Huggingface's transformers library, it is more convenient to directly use the transformers library.