# Model merge[[peft.utils.merge_utils.prune]]

PEFT provides several internal utilities for [merging LoRA adapters](../developer_guides/model_merging) with the TIES and DARE methods.

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>peft.utils.merge_utils.prune</name><anchor>peft.utils.merge_utils.prune</anchor><source>https://github.com/huggingface/peft/blob/v0.18.0.rc0/src/peft/utils/merge_utils.py#L75</source><parameters>[{"name": "tensor", "val": ": Tensor"}, {"name": "density", "val": ": float"}, {"name": "method", "val": ": typing.Literal['magnitude', 'random']"}, {"name": "rescale", "val": ": bool = False"}]</parameters><paramsdesc>- **tensor** (`torch.Tensor`) --The tensor to prune.
- **density** (`float`) --The fraction of values to preserve. Should be in [0,1].
- **method** (`str`) --The method to use to prune. Should be one of ["magnitude", "random"].
- **rescale** (`bool`) --Whether to rescale the result to preserve the expected value of the original tensor.</paramsdesc><paramgroups>0</paramgroups><rettype>`torch.Tensor`</rettype><retdesc>The pruned tensor.</retdesc></docstring>

Prune the values of task tensors based on the `method`.








</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>peft.utils.merge_utils.calculate_majority_sign_mask</name><anchor>peft.utils.merge_utils.calculate_majority_sign_mask</anchor><source>https://github.com/huggingface/peft/blob/v0.18.0.rc0/src/peft/utils/merge_utils.py#L103</source><parameters>[{"name": "tensor", "val": ": Tensor"}, {"name": "method", "val": ": typing.Literal['total', 'frequency'] = 'total'"}]</parameters><paramsdesc>- **tensor** (`torch.Tensor`) --The tensor to get the mask from.
- **method** (`str`) --The method to use to get the mask. Should be one of ["total", "frequency"].</paramsdesc><paramgroups>0</paramgroups><rettype>`torch.Tensor`</rettype><retdesc>The majority sign mask.</retdesc></docstring>

Get the mask of the majority sign across the task tensors. Task tensors are stacked on dimension 0.








</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>peft.utils.merge_utils.disjoint_merge</name><anchor>peft.utils.merge_utils.disjoint_merge</anchor><source>https://github.com/huggingface/peft/blob/v0.18.0.rc0/src/peft/utils/merge_utils.py#L128</source><parameters>[{"name": "task_tensors", "val": ": Tensor"}, {"name": "majority_sign_mask", "val": ": Tensor"}]</parameters><paramsdesc>- **task_tensors** (`torch.Tensor`) --The task tensors to merge.
- **majority_sign_mask** (`torch.Tensor`) --The mask of the majority sign across the task tensors.</paramsdesc><paramgroups>0</paramgroups><rettype>`torch.Tensor`</rettype><retdesc>The merged tensor.</retdesc></docstring>

Merge the task tensors using disjoint merge.








</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>peft.utils.merge_utils.task_arithmetic</name><anchor>peft.utils.merge_utils.task_arithmetic</anchor><source>https://github.com/huggingface/peft/blob/v0.18.0.rc0/src/peft/utils/merge_utils.py#L144</source><parameters>[{"name": "task_tensors", "val": ": list"}, {"name": "weights", "val": ": Tensor"}]</parameters><paramsdesc>- **task_tensors(`List[torch.Tensor]`)** --The task tensors to merge.
- **weights** (`torch.Tensor`) --The weights of the task tensors.</paramsdesc><paramgroups>0</paramgroups><rettype>`torch.Tensor`</rettype><retdesc>The merged tensor.</retdesc></docstring>

Merge the task tensors using `task arithmetic`.








</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>peft.utils.merge_utils.ties</name><anchor>peft.utils.merge_utils.ties</anchor><source>https://github.com/huggingface/peft/blob/v0.18.0.rc0/src/peft/utils/merge_utils.py#L185</source><parameters>[{"name": "task_tensors", "val": ": list"}, {"name": "weights", "val": ": Tensor"}, {"name": "density", "val": ": float"}, {"name": "majority_sign_method", "val": ": typing.Literal['total', 'frequency'] = 'total'"}]</parameters><paramsdesc>- **task_tensors(`List[torch.Tensor]`)** --The task tensors to merge.
- **weights** (`torch.Tensor`) --The weights of the task tensors.
- **density** (`float`) --The fraction of values to preserve. Should be in [0,1].
- **majority_sign_method** (`str`) --
  The method to use to get the majority sign mask. Should be one of ["total", "frequency"].</paramsdesc><paramgroups>0</paramgroups><rettype>`torch.Tensor`</rettype><retdesc>The merged tensor.</retdesc></docstring>

Merge the task tensors using `ties`.








</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>peft.utils.merge_utils.dare_linear</name><anchor>peft.utils.merge_utils.dare_linear</anchor><source>https://github.com/huggingface/peft/blob/v0.18.0.rc0/src/peft/utils/merge_utils.py#L217</source><parameters>[{"name": "task_tensors", "val": ": list"}, {"name": "weights", "val": ": Tensor"}, {"name": "density", "val": ": float"}]</parameters><paramsdesc>- **task_tensors(`List[torch.Tensor]`)** --The task tensors to merge.
- **weights** (`torch.Tensor`) --The weights of the task tensors.
- **density** (`float`) --The fraction of values to preserve. Should be in [0,1].</paramsdesc><paramgroups>0</paramgroups><rettype>`torch.Tensor`</rettype><retdesc>The merged tensor.</retdesc></docstring>

Merge the task tensors using `dare linear`.








</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>peft.utils.merge_utils.dare_ties</name><anchor>peft.utils.merge_utils.dare_ties</anchor><source>https://github.com/huggingface/peft/blob/v0.18.0.rc0/src/peft/utils/merge_utils.py#L239</source><parameters>[{"name": "task_tensors", "val": ": list"}, {"name": "weights", "val": ": Tensor"}, {"name": "density", "val": ": float"}, {"name": "majority_sign_method", "val": ": typing.Literal['total', 'frequency'] = 'total'"}]</parameters><paramsdesc>- **task_tensors(`List[torch.Tensor]`)** --The task tensors to merge.
- **weights** (`torch.Tensor`) --The weights of the task tensors.
- **density** (`float`) --The fraction of values to preserve. Should be in [0,1].
- **majority_sign_method** (`str`) --
  The method to use to get the majority sign mask. Should be one of ["total", "frequency"].</paramsdesc><paramgroups>0</paramgroups><rettype>`torch.Tensor`</rettype><retdesc>The merged tensor.</retdesc></docstring>

Merge the task tensors using `dare ties`.








</div>

<EditOnGithub source="https://github.com/huggingface/peft/blob/main/docs/source/package_reference/merge_utils.md" />