Present an API to Apache's DB files for mod_rewrite.
The process to update database files is complicated and hand-editing is strongly discouraged for the following reasons:
There did not appear to be a corruption free database format in
common between ruby and Apache that had a guaranteed consistent API. Even BerkeleyDB and the BDB module corrupted each other on testing.
Every effort was made to ensure that a crash, even due to a
system issue such as disk space or memory starvation did not result in a corrupt database and the loss of old information.
Every effort was made to ensure that multiple threads and
processes could not corrupt or step on each other.
While the httxt2dbm tool can run on an existing database, that
will result in additions but not removals from the database. Only some of your changes will take unless the entire db is recreated each time.
In order for BerkeleyDB to be safe for multiple processes to
access/edit, the environment must be specifically set up to allow locking. An audit of the Apache source code shows that it does not do that. And an strace of Apache shows no attempt to either lock or establish a mutex on the BerkeleyDB file. I believe the claim that BerkeleyDB is safe to have multiple processess reading/writing it is simply not true the way its used by Apache.
This locks down to one thread for safety. You MUST ensure that close is called to release all locks. Close also syncs changes to Apache if data was modified.
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 921 def initialize(flags=nil) @closed = false if self.MAPNAME.nil? raise NotImplementedError.new("Must subclass with proper map name.") end @config = ::OpenShift::Config.new @basedir = @config.get("OPENSHIFT_HTTP_CONF_DIR") @mode = 0640 if flags.nil? @flags = READER else @flags = flags end @filename = PathUtils.join(@basedir, self.MAPNAME) @lockfile = self.LOCKFILEBASE + '.' + self.MAPNAME + self.SUFFIX + '.lock' super() # Each filename needs its own mutex and lockfile self.LOCK.lock begin @lfd = File.new(@lockfile, Fcntl::O_RDWR | Fcntl::O_CREAT, 0640) @lfd.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) if writable? @lfd.flock(File::LOCK_EX) else @lfd.flock(File::LOCK_SH) end if @flags != NEWDB reload end rescue begin if not @lfd.nil? @lfd.close() end ensure self.LOCK.unlock end raise end end
Preferred method of access is to feed a block to open so we can guarantee the close.
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 1078 def self.open(flags=nil) inst = new(flags) if block_given? begin return yield(inst) rescue @flags = nil # Disable flush raise ensure if not inst.closed? inst.close end end end inst end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 1005 def callout # Use Berkeley DB so that there's no race condition between # multiple file moves. The Berkeley DB implementation creates a # scratch working file under certain circumstances. Use a # scratch dir to protect it. Dir.mktmpdir([File.basename(@filename) + ".db-", ""], File.dirname(@filename)) do |wd| tmpdb = PathUtils.join(wd, 'new.db') httxt2dbm = ["/usr/bin","/usr/sbin","/bin","/sbin"].map {|d| PathUtils.join(d, "httxt2dbm")}.select {|p| File.exists?(p)}.pop if httxt2dbm.nil? logger.warn("WARNING: no httxt2dbm command found, relying on PATH") httxt2dbm="httxt2dbm" end cmd = %Q{#{httxt2dbm} -f DB -i #{@filename}#{self.SUFFIX} -o #{tmpdb}} out,err,rc = ::OpenShift::Runtime::Utils::oo_spawn(cmd) if rc == 0 logger.debug("httxt2dbm: #{@filename}: #{rc}: stdout: #{out} stderr:#{err}") begin oldstat = File.stat(@filename + '.db') File.chown(oldstat.uid, oldstat.gid, tmpdb) File.chmod(oldstat.mode & 0777, tmpdb) rescue Errno::ENOENT end FileUtils.mv(tmpdb, @filename + '.db', :force=>true) else logger.error("ERROR: failure httxt2dbm #{@filename}: #{rc}: stdout: #{out} stderr:#{err}") unless rc == 0 end end end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 1059 def close @closed=true begin begin self.flush ensure @lfd.close() unless @lfd.closed? end ensure self.LOCK.unlock if self.LOCK.locked? end end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 1072 def closed? @closed end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 974 def decode_contents(f) f.each do |l| path, dest = l.strip.split if (not path.nil?) and (not dest.nil?) self.store(path, dest) end end end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 983 def encode_contents(f) self.each do |k, v| f.write([k, v].join(' ') + "\n") end end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 1036 def flush if writable? File.open(@filename + self.SUFFIX + '-', Fcntl::O_RDWR | Fcntl::O_CREAT | Fcntl::O_TRUNC, 0640) do |f| encode_contents(f) f.fsync end # Ruby 1.9 Hash preserves order, compare files to see if anything changed if FileUtils.compare_file(@filename + self.SUFFIX + '-', @filename + self.SUFFIX) FileUtils.rm(@filename + self.SUFFIX + '-', :force=>true) else begin oldstat = File.stat(@filename + self.SUFFIX) FileUtils.chown(oldstat.uid, oldstat.gid, @filename + self.SUFFIX + '-') FileUtils.chmod(oldstat.mode & 0777, @filename + self.SUFFIX + '-') rescue Errno::ENOENT end FileUtils.mv(@filename + self.SUFFIX + '-', @filename + self.SUFFIX, :force=>true) callout end end end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 989 def reload begin File.open(@filename + self.SUFFIX, Fcntl::O_RDONLY) do |f| decode_contents(f) end rescue Errno::ENOENT if not [WRCREAT, NEWDB].include?(@flags) raise end end end
# File lib/openshift-origin-node/model/frontend_httpd.rb, line 1001 def writable? [WRITER, WRCREAT, NEWDB].include?(@flags) end