HBase什么时候作minor major compact

HBase什么时候做minor major compact
我们都知道compact分为两类，一类叫Minor compact ,一类叫Major compact,
两者有什么区别呢？
两者的区别在于：Minor compact只是进行文件merge操作，而Major compact除了做文件Merge操作，还会将其中的delete项删除。
// TODO: Use better method for determining stamp of last major (HBASE-2990) long lowTimestamp = getLowestTimestamp(filesToCompact); long now = System.currentTimeMillis(); if (lowTimestamp > 0l && lowTimestamp < (now - mcTime)) { // Major compaction time has elapsed. if (filesToCompact.size() == 1) { // Single file StoreFile sf = filesToCompact.get(0); long oldest = (sf.getReader().timeRangeTracker == null) ? Long.MIN_VALUE : now - sf.getReader().timeRangeTracker.minimumTimestamp; if (sf.isMajorCompaction() && (this.ttl == HConstants.FOREVER || oldest < this.ttl)) { if (LOG.isDebugEnabled()) { LOG.debug("Skipping major compaction of " + this.storeNameStr + " because one (major) compacted file only and oldestTime " + oldest + "ms is < ttl=" + this.ttl); } } else if (this.ttl != HConstants.FOREVER && oldest > this.ttl) { LOG.debug("Major compaction triggered on store " + this.storeNameStr + ", because keyvalues outdated; time since last major compaction " + (now - lowTimestamp) + "ms"); result = true; } } else { if (LOG.isDebugEnabled()) { LOG.debug("Major compaction triggered on store " + this.storeNameStr + "; time since last major compaction " + (now - lowTimestamp) + "ms"); } result = true; } } return result; }

  long getNextMajorCompactTime() {    // default = 24hrs    long ret = conf.getLong(HConstants.MAJOR_COMPACTION_PERIOD, 1000*60*60*24);    if (family.getValue(HConstants.MAJOR_COMPACTION_PERIOD) != null) {      String strCompactionTime =        family.getValue(HConstants.MAJOR_COMPACTION_PERIOD);      ret = (new Long(strCompactionTime)).longValue();    }    if (ret > 0) {      // default = 20% = +/- 4.8 hrs      double jitterPct =  conf.getFloat("hbase.hregion.majorcompaction.jitter",          0.20F);      if (jitterPct > 0) {        long jitter = Math.round(ret * jitterPct);        // deterministic jitter avoids a major compaction storm on restart        ImmutableList<StoreFile> snapshot = storefiles;         if (snapshot != null && !snapshot.isEmpty()) {          String seed = snapshot.get(0).getPath().getName();          double curRand = new Random(seed.hashCode()).nextDouble();          ret += jitter - Math.round(2L * jitter * curRand);        } else {          ret = 0; // no storefiles == no major compaction        }      }    }    return ret;  }

2. 在compact时，当选中的文件数小于这个store所有文件数*ratio(hbase.hstore.compaction.ratio)时，进行Minor compact.
当

boolean isMajor = (filesToCompact.getFilesToCompact().size() == this.storefiles.size());

时，也就是当选中的文件数等于该store数时，会进行Major compact.

查看更多 下一篇