Class GC
- java.lang.Object
-
- org.eclipse.jgit.internal.storage.file.GC
-
public class GC extends java.lang.Object
A garbage collector for gitFileRepository
. Instances of this class are not thread-safe. Don't use the same instance from multiple threads. This class started as a copy of DfsGarbageCollector from Shawn O. Pearce adapted to FileRepositories.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
GC.RepoStatistics
A class holding statistical data for a FileRepository regarding how many objects are stored as loose or packed objects
-
Field Summary
Fields Modifier and Type Field Description private boolean
automatic
Whether gc should do automatic housekeepingprivate boolean
background
Whether to run gc in a background threadprivate static java.lang.String
BITMAP_EXT
private static int
DEFAULT_AUTOLIMIT
private static int
DEFAULT_AUTOPACKLIMIT
private static java.util.concurrent.ExecutorService
executor
private java.util.Date
expire
private long
expireAgeMillis
private static java.lang.String
INDEX_EXT
private static java.lang.String
KEEP_EXT
private java.util.Collection<Ref>
lastPackedRefs
the refs which existed during the last call torepack()
.private long
lastRepackTime
Holds the starting time of the last repack() execution.private static org.slf4j.Logger
LOG
private static java.lang.String
PACK_EXT
private java.util.Date
packExpire
private long
packExpireAgeMillis
private static java.util.regex.Pattern
PATTERN_LOOSE_OBJECT
private PackConfig
pconfig
private ProgressMonitor
pm
private static java.lang.String
PRUNE_EXPIRE_DEFAULT
private static java.lang.String
PRUNE_PACK_EXPIRE_DEFAULT
private FileRepository
repo
-
Constructor Summary
Constructors Constructor Description GC(FileRepository repo)
Creates a new garbage collector with default values.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private void
addRepackAllOption()
private boolean
canBeSafelyDeleted(java.nio.file.Path path, java.time.Instant threshold)
private void
checkCancelled()
private void
delete(java.nio.file.Path d)
private void
deleteDir(java.nio.file.Path dir)
private void
deleteEmptyRefsFolders()
private void
deleteOldPacks(java.util.Collection<Pack> oldPacks, java.util.Collection<Pack> newPacks)
Delete old pack files.private void
deleteOrphans()
Deletes orphansprivate void
deleteTempPacksIdx()
private java.util.Collection<Pack>
doGc()
private static boolean
equals(Ref r1, Ref r2)
private java.util.concurrent.ExecutorService
executor()
java.util.Collection<Pack>
gc()
Runs a garbage collector on aFileRepository
.private java.util.Collection<Ref>
getAllRefs()
Returns a collection of all refs and additional refs.private long
getExpireDate()
private int
getLooseObjectLimit()
private long
getPackExpireDate()
private java.lang.String
getPruneExpireStr()
GC.RepoStatistics
getStatistics()
Returns information about objects and pack files for a FileRepository.private boolean
isDirectory(java.nio.file.Path p)
private static boolean
isHead(Ref ref)
private static boolean
isTag(Ref ref)
private java.util.Set<ObjectId>
listNonHEADIndexObjects()
Return a list of those objects in the index which differ from whats in HEADprivate java.util.Set<ObjectId>
listRefLogObjects(Ref ref, long minTime)
private void
loosen(ObjectDirectoryInserter inserter, ObjectReader reader, Pack pack, java.util.HashSet<ObjectId> existing)
Loosen objects in a pack file which are not also in the newly-created pack files.private boolean
needGc()
void
packRefs()
Pack ref storage.void
prune(java.util.Set<ObjectId> objectsToKeep)
Like "git prune" this method tries to prune all loose objects which are unreferenced.private void
prunePack(PackFile packFile)
Delete files associated with a single pack file.void
prunePacked()
Like "git prune-packed" this method tries to prune all loose objects which can be found in packs.private void
prunePreserved()
Delete the preserved directory including all pack files withinprivate void
removeOldPack(PackFile packFile, int deleteOptions)
Deletes old pack file, unless 'preserve-oldpacks' is set, in which case it moves the pack file to the preserved directoryprivate void
removeReferenced(java.util.Map<ObjectId,java.io.File> id2File, ObjectWalk w)
Remove all entries from a map which key is the id of an object referenced by the given ObjectWalkjava.util.Collection<Pack>
repack()
Packs all objects which reachable from any of the heads into one pack file.void
setAuto(boolean auto)
Set thegc --auto
option.(package private) void
setBackground(boolean background)
static void
setExecutor(java.util.concurrent.ExecutorService e)
Set the executor for running auto-gc in the background.void
setExpire(java.util.Date expire)
During gc() or prune() each unreferenced, loose object which has been created or modified after or atexpire
will not be pruned.void
setExpireAgeMillis(long expireAgeMillis)
During gc() or prune() each unreferenced, loose object which has been created or modified in the lastexpireAgeMillis
milliseconds will not be pruned.void
setPackConfig(PackConfig pconfig)
Set the PackConfig used when (re-)writing packfiles.void
setPackExpire(java.util.Date packExpire)
During gc() or prune() packfiles which are created or modified after or atpackExpire
will not be deleted.void
setPackExpireAgeMillis(long packExpireAgeMillis)
During gc() or prune() packfiles which are created or modified in the lastpackExpireAgeMillis
milliseconds will not be deleted.GC
setProgressMonitor(ProgressMonitor pm)
Set the progress monitor used for garbage collection methods.(package private) boolean
tooManyLooseObjects()
Quickly estimate number of loose objects, SHA1 is distributed evenly so counting objects in one directory (bucket 17) is sufficient(package private) boolean
tooManyPacks()
private Pack
writePack(java.util.Set<? extends ObjectId> want, java.util.Set<? extends ObjectId> have, java.util.Set<ObjectId> tags, java.util.Set<ObjectId> tagTargets, java.util.List<ObjectIdSet> excludeObjects)
-
-
-
Field Detail
-
LOG
private static final org.slf4j.Logger LOG
-
PRUNE_EXPIRE_DEFAULT
private static final java.lang.String PRUNE_EXPIRE_DEFAULT
- See Also:
- Constant Field Values
-
PRUNE_PACK_EXPIRE_DEFAULT
private static final java.lang.String PRUNE_PACK_EXPIRE_DEFAULT
- See Also:
- Constant Field Values
-
PATTERN_LOOSE_OBJECT
private static final java.util.regex.Pattern PATTERN_LOOSE_OBJECT
-
PACK_EXT
private static final java.lang.String PACK_EXT
-
BITMAP_EXT
private static final java.lang.String BITMAP_EXT
-
INDEX_EXT
private static final java.lang.String INDEX_EXT
-
KEEP_EXT
private static final java.lang.String KEEP_EXT
-
DEFAULT_AUTOPACKLIMIT
private static final int DEFAULT_AUTOPACKLIMIT
- See Also:
- Constant Field Values
-
DEFAULT_AUTOLIMIT
private static final int DEFAULT_AUTOLIMIT
- See Also:
- Constant Field Values
-
executor
private static volatile java.util.concurrent.ExecutorService executor
-
repo
private final FileRepository repo
-
pm
private ProgressMonitor pm
-
expireAgeMillis
private long expireAgeMillis
-
expire
private java.util.Date expire
-
packExpireAgeMillis
private long packExpireAgeMillis
-
packExpire
private java.util.Date packExpire
-
pconfig
private PackConfig pconfig
-
lastPackedRefs
private java.util.Collection<Ref> lastPackedRefs
the refs which existed during the last call torepack()
. This is needed duringprune(Set)
where we can optimize by looking at the difference between the current refs and the refs which existed during lastrepack()
.
-
lastRepackTime
private long lastRepackTime
Holds the starting time of the last repack() execution. This is needed in prune() to inspect only those reflog entries which have been added since last repack().
-
automatic
private boolean automatic
Whether gc should do automatic housekeeping
-
background
private boolean background
Whether to run gc in a background thread
-
-
Constructor Detail
-
GC
public GC(FileRepository repo)
Creates a new garbage collector with default values. An expirationTime of two weeks andnull
as progress monitor will be used.- Parameters:
repo
- the repo to work on
-
-
Method Detail
-
setExecutor
public static void setExecutor(java.util.concurrent.ExecutorService e)
Set the executor for running auto-gc in the background. If no executor is set JGit's own WorkQueue will be used.- Parameters:
e
- the executor to be used for running auto-gc
-
gc
public java.util.Collection<Pack> gc() throws java.io.IOException, java.text.ParseException
Runs a garbage collector on aFileRepository
. It will- pack loose references into packed-refs
- repack all reachable objects into new pack files and delete the old pack files
- prune all loose objects which are now reachable by packs
setAuto(boolean)
was set totrue
gc
will first check whether any housekeeping is required; if not, it exits without performing any work. IfsetBackground(boolean)
was set totrue
collectGarbage
will start the gc in the background, and then return immediately. In this case, errors will not be reported except in gc.log.- Returns:
- the collection of
Pack
's which are newly created - Throws:
java.io.IOException
java.text.ParseException
- If the configuration parameter "gc.pruneexpire" couldn't be parsed
-
executor
private java.util.concurrent.ExecutorService executor()
-
doGc
private java.util.Collection<Pack> doGc() throws java.io.IOException, java.text.ParseException
- Throws:
java.io.IOException
java.text.ParseException
-
loosen
private void loosen(ObjectDirectoryInserter inserter, ObjectReader reader, Pack pack, java.util.HashSet<ObjectId> existing) throws java.io.IOException
Loosen objects in a pack file which are not also in the newly-created pack files.- Parameters:
inserter
-reader
-pack
-existing
-- Throws:
java.io.IOException
-
deleteOldPacks
private void deleteOldPacks(java.util.Collection<Pack> oldPacks, java.util.Collection<Pack> newPacks) throws java.text.ParseException, java.io.IOException
Delete old pack files. What is 'old' is defined by specifying a set of old pack files and a set of new pack files. Each pack file contained in old pack files but not contained in new pack files will be deleted. If preserveOldPacks is set, keep a copy of the pack file in the preserve directory. If an expirationDate is set then pack files which are younger than the expirationDate will not be deleted nor preserved.If we're not immediately expiring loose objects, loosen any objects in the old pack files which aren't in the new pack files.
- Parameters:
oldPacks
-newPacks
-- Throws:
java.text.ParseException
java.io.IOException
-
removeOldPack
private void removeOldPack(PackFile packFile, int deleteOptions) throws java.io.IOException
Deletes old pack file, unless 'preserve-oldpacks' is set, in which case it moves the pack file to the preserved directory- Parameters:
packFile
-deleteOptions
-- Throws:
java.io.IOException
-
prunePreserved
private void prunePreserved()
Delete the preserved directory including all pack files within
-
prunePack
private void prunePack(PackFile packFile)
Delete files associated with a single pack file. First try to delete the ".pack" file because on some platforms the ".pack" file may be locked and can't be deleted. In such a case it is better to detect this early and give up on deleting files for this packfile. Otherwise we may delete the ".index" file and when failing to delete the ".pack" file we are left with a ".pack" file without a ".index" file.- Parameters:
packFile
-
-
prunePacked
public void prunePacked() throws java.io.IOException
Like "git prune-packed" this method tries to prune all loose objects which can be found in packs. If certain objects can't be pruned (e.g. because the filesystem delete operation fails) this is silently ignored.- Throws:
java.io.IOException
-
prune
public void prune(java.util.Set<ObjectId> objectsToKeep) throws java.io.IOException, java.text.ParseException
Like "git prune" this method tries to prune all loose objects which are unreferenced. If certain objects can't be pruned (e.g. because the filesystem delete operation fails) this is silently ignored.- Parameters:
objectsToKeep
- a set of objects which should explicitly not be pruned- Throws:
java.io.IOException
java.text.ParseException
- If the configuration parameter "gc.pruneexpire" couldn't be parsed
-
getExpireDate
private long getExpireDate() throws java.text.ParseException
- Throws:
java.text.ParseException
-
getPruneExpireStr
private java.lang.String getPruneExpireStr()
-
getPackExpireDate
private long getPackExpireDate() throws java.text.ParseException
- Throws:
java.text.ParseException
-
removeReferenced
private void removeReferenced(java.util.Map<ObjectId,java.io.File> id2File, ObjectWalk w) throws MissingObjectException, IncorrectObjectTypeException, java.io.IOException
Remove all entries from a map which key is the id of an object referenced by the given ObjectWalk- Parameters:
id2File
-w
-- Throws:
MissingObjectException
IncorrectObjectTypeException
java.io.IOException
-
packRefs
public void packRefs() throws java.io.IOException
Pack ref storage. For a RefDirectory database, this packs all non-symbolic, loose refs into packed-refs. For Reftable, all of the data is compacted into a single table.- Throws:
java.io.IOException
-
repack
public java.util.Collection<Pack> repack() throws java.io.IOException
Packs all objects which reachable from any of the heads into one pack file. Additionally all objects which are not reachable from any head but which are reachable from any of the other refs (e.g. tags), special refs (e.g. FETCH_HEAD) or index are packed into a separate pack file. Objects included in pack files which have a .keep file associated are never repacked. All old pack files which existed before are deleted.- Returns:
- a collection of the newly created pack files
- Throws:
java.io.IOException
- when during reading of refs, index, packfiles, objects, reflog-entries or during writing to the packfilesIOException
occurs
-
isHead
private static boolean isHead(Ref ref)
-
isTag
private static boolean isTag(Ref ref)
-
deleteEmptyRefsFolders
private void deleteEmptyRefsFolders() throws java.io.IOException
- Throws:
java.io.IOException
-
canBeSafelyDeleted
private boolean canBeSafelyDeleted(java.nio.file.Path path, java.time.Instant threshold)
-
deleteDir
private void deleteDir(java.nio.file.Path dir)
-
isDirectory
private boolean isDirectory(java.nio.file.Path p)
-
delete
private void delete(java.nio.file.Path d)
-
deleteOrphans
private void deleteOrphans()
Deletes orphansA file is considered an orphan if it is either a "bitmap" or an index file, and its corresponding pack file is missing in the list.
-
deleteTempPacksIdx
private void deleteTempPacksIdx()
-
listRefLogObjects
private java.util.Set<ObjectId> listRefLogObjects(Ref ref, long minTime) throws java.io.IOException
- Parameters:
ref
- the ref which log should be inspectedminTime
- only reflog entries not older then this time are processed- Returns:
- the
ObjectId
s contained in the reflog - Throws:
java.io.IOException
-
getAllRefs
private java.util.Collection<Ref> getAllRefs() throws java.io.IOException
Returns a collection of all refs and additional refs. Additional refs which don't start with "refs/" are not returned because they should not save objects from being garbage collected. Examples for such references are ORIG_HEAD, MERGE_HEAD, FETCH_HEAD and CHERRY_PICK_HEAD.- Returns:
- a collection of refs pointing to live objects.
- Throws:
java.io.IOException
-
listNonHEADIndexObjects
private java.util.Set<ObjectId> listNonHEADIndexObjects() throws CorruptObjectException, java.io.IOException
Return a list of those objects in the index which differ from whats in HEAD- Returns:
- a set of ObjectIds of changed objects in the index
- Throws:
java.io.IOException
CorruptObjectException
NoWorkTreeException
-
writePack
private Pack writePack(@NonNull java.util.Set<? extends ObjectId> want, @NonNull java.util.Set<? extends ObjectId> have, @NonNull java.util.Set<ObjectId> tags, java.util.Set<ObjectId> tagTargets, java.util.List<ObjectIdSet> excludeObjects) throws java.io.IOException
- Throws:
java.io.IOException
-
checkCancelled
private void checkCancelled() throws CancelledException
- Throws:
CancelledException
-
getStatistics
public GC.RepoStatistics getStatistics() throws java.io.IOException
Returns information about objects and pack files for a FileRepository.- Returns:
- information about objects and pack files for a FileRepository
- Throws:
java.io.IOException
-
setProgressMonitor
public GC setProgressMonitor(ProgressMonitor pm)
Set the progress monitor used for garbage collection methods.- Parameters:
pm
- aProgressMonitor
object.- Returns:
- this
-
setExpireAgeMillis
public void setExpireAgeMillis(long expireAgeMillis)
During gc() or prune() each unreferenced, loose object which has been created or modified in the lastexpireAgeMillis
milliseconds will not be pruned. Only older objects may be pruned. If set to 0 then every object is a candidate for pruning.- Parameters:
expireAgeMillis
- minimal age of objects to be pruned in milliseconds.
-
setPackExpireAgeMillis
public void setPackExpireAgeMillis(long packExpireAgeMillis)
During gc() or prune() packfiles which are created or modified in the lastpackExpireAgeMillis
milliseconds will not be deleted. Only older packfiles may be deleted. If set to 0 then every packfile is a candidate for deletion.- Parameters:
packExpireAgeMillis
- minimal age of packfiles to be deleted in milliseconds.
-
setPackConfig
public void setPackConfig(@NonNull PackConfig pconfig)
Set the PackConfig used when (re-)writing packfiles. This allows to influence how packs are written and to implement something similar to "git gc --aggressive"- Parameters:
pconfig
- thePackConfig
used when writing packs
-
setExpire
public void setExpire(java.util.Date expire)
During gc() or prune() each unreferenced, loose object which has been created or modified after or atexpire
will not be pruned. Only older objects may be pruned. If set to null then every object is a candidate for pruning.- Parameters:
expire
- instant in time which defines object expiration objects with modification time before this instant are expired objects with modification time newer or equal to this instant are not expired
-
setPackExpire
public void setPackExpire(java.util.Date packExpire)
During gc() or prune() packfiles which are created or modified after or atpackExpire
will not be deleted. Only older packfiles may be deleted. If set to null then every packfile is a candidate for deletion.- Parameters:
packExpire
- instant in time which defines packfile expiration
-
setAuto
public void setAuto(boolean auto)
Set thegc --auto
option. With this option, gc checks whether any housekeeping is required; if not, it exits without performing any work. Some JGit commands rungc --auto
after performing operations that could create many loose objects.Housekeeping is required if there are too many loose objects or too many packs in the repository. If the number of loose objects exceeds the value of the gc.auto option JGit GC consolidates all existing packs into a single pack (equivalent to
-A
option), whereas git-core would combine all loose objects into a single pack usingrepack -d -l
. Setting the value ofgc.auto
to 0 disables automatic packing of loose objects.If the number of packs exceeds the value of
gc.autoPackLimit
, then existing packs (except those marked with a .keep file) are consolidated into a single pack by using the-A
option of repack. Settinggc.autoPackLimit
to 0 disables automatic consolidation of packs.Like git the following jgit commands run auto gc:
- fetch
- merge
- rebase
- receive-pack
receive.autogc = false
- Parameters:
auto
- defines whether gc should do automatic housekeeping
-
setBackground
void setBackground(boolean background)
- Parameters:
background
- whether to run the gc in a background thread.
-
needGc
private boolean needGc()
-
addRepackAllOption
private void addRepackAllOption()
-
tooManyPacks
boolean tooManyPacks()
- Returns:
true
if number of packs > gc.autopacklimit (default 50)
-
tooManyLooseObjects
boolean tooManyLooseObjects()
Quickly estimate number of loose objects, SHA1 is distributed evenly so counting objects in one directory (bucket 17) is sufficient- Returns:
true
if number of loose objects > gc.auto (default 6700)
-
getLooseObjectLimit
private int getLooseObjectLimit()
-
-