class Dalli::Client

Public Class Methods

new(servers=nil, options={}) click to toggle source

Dalli::Client is the main class which developers will use to interact with the memcached server. Usage:

Dalli::Client.new(['localhost:11211:10', 'cache-2.example.com:11211:5', '192.168.0.1:22122:5', '/var/run/memcached/socket'],
                :threadsafe => true, :failover => true, :expires_in => 300)

servers is an Array of “host:port:weight” where weight allows you to distribute cache unevenly. Both weight and port are optional. If you pass in nil, Dalli will use the MEMCACHE_SERVERS environment variable or default to 'localhost:11211' if it is not present. Dalli also supports the ability to connect to Memcached on localhost through a UNIX socket. To use this functionality, use a full pathname (beginning with a slash character '/') in place of the “host:port” pair in the server configuration.

Options:

  • :namespace - prepend each key with this value to provide simple namespacing.

  • :failover - if a server is down, look for and store values on another server in the ring. Default: true.

  • :threadsafe - ensure that only one thread is actively using a socket at a time. Default: true.

  • :expires_in - default TTL in seconds if you do not pass TTL as a parameter to an individual operation, defaults to 0 or forever

  • :compress - defaults to false, if true Dalli will compress values larger than 1024 bytes before sending them to memcached.

  • :serializer - defaults to Marshal

  • :compressor - defaults to zlib

# File lib/dalli/client.rb, line 31
def initialize(servers=nil, options={})
  @servers = normalize_servers(servers || ENV["MEMCACHE_SERVERS"] || '127.0.0.1:11211')
  @options = normalize_options(options)
  @ring = nil
end

Public Instance Methods

add(key, value, ttl=nil, options=nil) click to toggle source

Conditionally add a key/value pair, if the key does not already exist on the server. Returns truthy if the operation succeeded.

# File lib/dalli/client.rb, line 113
def add(key, value, ttl=nil, options=nil)
  ttl ||= @options[:expires_in].to_i
  perform(:add, key, value, ttl, options)
end
alive!() click to toggle source

Make sure memcache servers are alive, or raise an Dalli::RingError

# File lib/dalli/client.rb, line 221
def alive!
  ring.server_for_key("")
end
append(key, value) click to toggle source

Append value to the value already stored on the server for 'key'. Appending only works for values stored with :raw => true.

# File lib/dalli/client.rb, line 133
def append(key, value)
  perform(:append, key, value.to_s)
end
cas(key, ttl=nil, options=nil) { |value| ... } click to toggle source

compare and swap values using optimistic locking. Fetch the existing value for key. If it exists, yield the value to the block. Add the block's return value as the new value for the key. Add will fail if someone else changed the value.

Returns:

  • nil if the key did not exist.

  • false if the value was changed by someone else.

  • true if the value was successfully updated.

# File lib/dalli/client.rb, line 95
def cas(key, ttl=nil, options=nil)
  ttl ||= @options[:expires_in].to_i
  (value, cas) = perform(:cas, key)
  value = (!value || value == 'Not found') ? nil : value
  if value
    newvalue = yield(value)
    perform(:set, key, newvalue, ttl, cas, options)
  end
end
close() click to toggle source

Close our connection to each server. If you perform another operation after this, the connections will be re-established.

# File lib/dalli/client.rb, line 238
def close
  if @ring
    @ring.servers.each { |s| s.close }
    @ring = nil
  end
end
Also aliased as: reset
decr(key, amt=1, ttl=nil, default=nil) click to toggle source

Decr subtracts the given amount from the counter on the memcached server. Amt must be a positive integer value.

memcached counters are unsigned and cannot hold negative values. Calling decr on a counter which is 0 will just return 0.

If default is nil, the counter must already exist or the operation will fail and will return nil. Otherwise this method will return the new value for the counter.

Note that the ttl will only apply if the counter does not already exist. To decrease an existing counter and update its TTL, use cas.

# File lib/dalli/client.rb, line 182
def decr(key, amt=1, ttl=nil, default=nil)
  raise ArgumentError, "Positive values only: #{amt}" if amt < 0
  ttl ||= @options[:expires_in].to_i
  perform(:decr, key, amt.to_i, ttl, default)
end
delete(key) click to toggle source
# File lib/dalli/client.rb, line 126
def delete(key)
  perform(:delete, key, 0)
end
delete_cas(key, cas=0) click to toggle source

Delete a key/value pair, verifying existing CAS. Returns true if succeeded, and falsy otherwise.

# File lib/dalli/cas/client.rb, line 53
def delete_cas(key, cas=0)
  perform(:delete, key, cas)
end
fetch(key, ttl=nil, options=nil) { || ... } click to toggle source
# File lib/dalli/client.rb, line 74
def fetch(key, ttl=nil, options=nil)
  ttl ||= @options[:expires_in].to_i
  val = get(key, options)
  if val.nil? && block_given?
    val = yield
    add(key, val, ttl, options)
  end
  val
end
flush(delay=0) click to toggle source
# File lib/dalli/client.rb, line 144
def flush(delay=0)
  time = -delay
  ring.servers.map { |s| s.request(:flush, time += delay) }
end
Also aliased as: flush_all
flush_all(delay=0)
Alias for: flush
get(key, options=nil) click to toggle source

Get the value associated with the key.

# File lib/dalli/client.rb, line 55
def get(key, options=nil)
  perform(:get, key)
end
get_cas(key) { |value, cas| ... } click to toggle source

Get the value and CAS ID associated with the key. If a block is provided, value and CAS will be passed to the block.

# File lib/dalli/cas/client.rb, line 8
def get_cas(key)
  (value, cas) = perform(:cas, key)
  value = (!value || value == 'Not found') ? nil : value
  if block_given?
    yield value, cas
  else
    [value, cas]
  end
end
get_multi(*keys) { |k, first| ... } click to toggle source

Fetch multiple keys efficiently. If a block is given, yields key/value pairs one at a time. Otherwise returns a hash of { 'key' => 'value', 'key2' => 'value1' }

# File lib/dalli/client.rb, line 63
def get_multi(*keys)
  return {} if keys.flatten.compact.empty?
  if block_given?
    get_multi_yielder(keys) {|k, data| yield k, data.first}
  else
    Hash.new.tap do |hash|
      get_multi_yielder(keys) {|k, data| hash[k] = data.first}
    end
  end
end
get_multi_cas(*keys) { |*args| ... } click to toggle source

Fetch multiple keys efficiently, including available metadata such as CAS. If a block is given, yields key/data pairs one a time. Data is an array:

value, cas_id

If no block is given, returns a hash of

{ 'key' => [value, cas_id] }
# File lib/dalli/cas/client.rb, line 24
def get_multi_cas(*keys)
  if block_given?
    get_multi_yielder(keys) {|*args| yield(*args)}
  else
    Hash.new.tap do |hash|
      get_multi_yielder(keys) {|k, data| hash[k] = data}
    end
  end
end
incr(key, amt=1, ttl=nil, default=nil) click to toggle source

Incr adds the given amount to the counter on the memcached server. Amt must be a positive integer value.

If default is nil, the counter must already exist or the operation will fail and will return nil. Otherwise this method will return the new value for the counter.

Note that the ttl will only apply if the counter does not already exist. To increase an existing counter and update its TTL, use cas.

# File lib/dalli/client.rb, line 162
def incr(key, amt=1, ttl=nil, default=nil)
  raise ArgumentError, "Positive values only: #{amt}" if amt < 0
  ttl ||= @options[:expires_in].to_i
  perform(:incr, key, amt.to_i, ttl, default)
end
multi() { || ... } click to toggle source

Turn on quiet aka noreply support. All relevant operations within this block will be effectively pipelined as Dalli will use 'quiet' operations where possible. Currently supports the set, add, replace and delete operations.

# File lib/dalli/client.rb, line 46
def multi
  old, Thread.current[:dalli_multi] = Thread.current[:dalli_multi], true
  yield
ensure
  Thread.current[:dalli_multi] = old
end
prepend(key, value) click to toggle source

Prepend value to the value already stored on the server for 'key'. Prepending only works for values stored with :raw => true.

# File lib/dalli/client.rb, line 140
def prepend(key, value)
  perform(:prepend, key, value.to_s)
end
replace(key, value, ttl=nil, options=nil) click to toggle source

Conditionally add a key/value pair, only if the key already exists on the server. Returns truthy if the operation succeeded.

# File lib/dalli/client.rb, line 121
def replace(key, value, ttl=nil, options=nil)
  ttl ||= @options[:expires_in].to_i
  perform(:replace, key, value, ttl, 0, options)
end
replace_cas(key, value, cas, ttl=nil, options=nil) click to toggle source

Conditionally add a key/value pair, verifying existing CAS, only if the key already exists on the server. Returns the new CAS value if the operation succeeded, or falsy otherwise.

# File lib/dalli/cas/client.rb, line 46
def replace_cas(key, value, cas, ttl=nil, options=nil)
  ttl ||= @options[:expires_in].to_i
  perform(:replace, key, value, ttl, cas, options)
end
reset()
Alias for: close
reset_stats() click to toggle source

Reset stats for each server.

# File lib/dalli/client.rb, line 213
def reset_stats
  ring.servers.map do |server|
    server.alive? ? server.request(:reset_stats) : nil
  end
end
set(key, value, ttl=nil, options=nil) click to toggle source
# File lib/dalli/client.rb, line 105
def set(key, value, ttl=nil, options=nil)
  ttl ||= @options[:expires_in].to_i
  perform(:set, key, value, ttl, 0, options)
end
set_cas(key, value, cas, ttl=nil, options=nil) click to toggle source

Set the key-value pair, verifying existing CAS. Returns the resulting CAS value if succeeded, and falsy otherwise.

# File lib/dalli/cas/client.rb, line 37
def set_cas(key, value, cas, ttl=nil, options=nil)
  ttl ||= @options[:expires_in].to_i
  perform(:set, key, value, ttl, cas, options)
end
stats(type=nil) click to toggle source

Collect the stats for each server. You can optionally pass a type including :items, :slabs or :settings to get specific stats Returns a hash like { 'hostname:port' => { 'stat1' => 'value1', … }, 'hostname2:port' => { … } }

# File lib/dalli/client.rb, line 202
def stats(type=nil)
  type = nil if ![nil, :items,:slabs,:settings].include? type
  values = {}
  ring.servers.each do |server|
    values["#{server.name}"] = server.alive? ? server.request(:stats,type.to_s) : nil
  end
  values
end
touch(key, ttl=nil) click to toggle source

Touch updates expiration time for a given key.

Returns true if key exists, otherwise nil.

# File lib/dalli/client.rb, line 192
def touch(key, ttl=nil)
  ttl ||= @options[:expires_in].to_i
  resp = perform(:touch, key, ttl)
  resp.nil? ? nil : true
end
version() click to toggle source

Version of the memcache servers.

# File lib/dalli/client.rb, line 227
def version
  values = {}
  ring.servers.each do |server|
    values["#{server.name}"] = server.alive? ? server.request(:version) : nil
  end
  values
end
with() { |self| ... } click to toggle source

Stub method so a bare Dalli client can pretend to be a connection pool.

# File lib/dalli/client.rb, line 247
def with
  yield self
end

Private Instance Methods

get_multi_yielder(keys) { |key_without_namespace(key), value_list| ... } click to toggle source

Yields, one at a time, keys and their values+attributes.

# File lib/dalli/client.rb, line 380
def get_multi_yielder(keys)
  perform do
    return {} if keys.empty?
    ring.lock do
      begin
        groups = groups_for_keys(keys)
        if unfound_keys = groups.delete(nil)
          Dalli.logger.debug { "unable to get keys for #{unfound_keys.length} keys because no matching server was found" }
        end
        make_multi_get_requests(groups)

        servers = groups.keys
        return if servers.empty?
        servers = perform_multi_response_start(servers)

        start = Time.now
        loop do
          # remove any dead servers
          servers.delete_if { |s| s.sock.nil? }
          break if servers.empty?

          # calculate remaining timeout
          elapsed = Time.now - start
          timeout = servers.first.options[:socket_timeout]
          if elapsed > timeout
            readable = nil
          else
            sockets = servers.map(&:sock)
            readable, _ = IO.select(sockets, nil, nil, timeout - elapsed)
          end

          if readable.nil?
            # no response within timeout; abort pending connections
            servers.each do |server|
              Dalli.logger.debug { "memcached at #{server.name} did not response within timeout" }
              server.multi_response_abort
            end
            break

          else
            readable.each do |sock|
              server = sock.server

              begin
                server.multi_response_nonblock.each_pair do |key, value_list|
                  yield key_without_namespace(key), value_list
                end

                if server.multi_response_completed?
                  servers.delete(server)
                end
              rescue NetworkError
                servers.delete(server)
              end
            end
          end
        end
      end
    end
  end
end
groups_for_keys(*keys) click to toggle source
# File lib/dalli/client.rb, line 253
def groups_for_keys(*keys)
  groups = mapped_keys(keys).flatten.group_by do |key|
    begin
      ring.server_for_key(key)
    rescue Dalli::RingError
      Dalli.logger.debug { "unable to get key #{key}" }
      nil
    end
  end
  return groups
end
key_with_namespace(key) click to toggle source
# File lib/dalli/client.rb, line 352
def key_with_namespace(key)
  (ns = namespace) ? "#{ns}:#{key}" : key
end
key_without_namespace(key) click to toggle source
# File lib/dalli/client.rb, line 356
def key_without_namespace(key)
  (ns = namespace) ? key.sub(%r(\A#{ns}:), '') : key
end
make_multi_get_requests(groups) click to toggle source
# File lib/dalli/client.rb, line 269
def make_multi_get_requests(groups)
  groups.each do |server, keys_for_server|
    begin
      # TODO: do this with the perform chokepoint?
      # But given the fact that fetching the response doesn't take place
      # in that slot it's misleading anyway. Need to move all of this method
      # into perform to be meaningful
      server.request(:send_multiget, keys_for_server)
    rescue DalliError, NetworkError => e
      Dalli.logger.debug { e.inspect }
      Dalli.logger.debug { "unable to get keys for server #{server.name}" }
    end
  end
end
mapped_keys(keys) click to toggle source
# File lib/dalli/client.rb, line 265
def mapped_keys(keys)
  keys.flatten.map {|a| validate_key(a.to_s)}
end
namespace() click to toggle source
# File lib/dalli/client.rb, line 360
def namespace
  return nil unless @options[:namespace]
  @options[:namespace].is_a?(Proc) ? @options[:namespace].call.to_s : @options[:namespace].to_s
end
normalize_options(opts) click to toggle source
# File lib/dalli/client.rb, line 365
def normalize_options(opts)
  if opts[:compression]
    Dalli.logger.warn "DEPRECATED: Dalli's :compression option is now just :compress => true.  Please update your configuration."
    opts[:compress] = opts.delete(:compression)
  end
  begin
    opts[:expires_in] = opts[:expires_in].to_i if opts[:expires_in]
  rescue NoMethodError
    raise ArgumentError, "cannot convert :expires_in => #{opts[:expires_in].inspect} to an integer"
  end
  opts
end
normalize_servers(servers) click to toggle source

Normalizes the argument into an array of servers. If the argument is a string, it's expected to be of the format “memcache1.example.com:11211[,memcache2.example.com:11211[,memcache3.example.com:11211]]

# File lib/dalli/client.rb, line 301
def normalize_servers(servers)
  if servers.is_a? String
    return servers.split(",")
  else
    return servers
  end
end
perform(*all_args, &blk) click to toggle source

Chokepoint method for instrumentation

# File lib/dalli/client.rb, line 325
def perform(*all_args, &blk)
  return blk.call if blk
  op, key, *args = *all_args

  key = key.to_s
  key = validate_key(key)
  begin
    server = ring.server_for_key(key)
    ret = server.request(op, key, *args)
    ret
  rescue NetworkError => e
    Dalli.logger.debug { e.inspect }
    Dalli.logger.debug { "retrying request with new server" }
    retry
  end
end
perform_multi_response_start(servers) click to toggle source
# File lib/dalli/client.rb, line 284
def perform_multi_response_start(servers)
  servers.each do |server|
    next unless server.alive?
    begin
      server.multi_response_start
    rescue DalliError, NetworkError => e
      Dalli.logger.debug { e.inspect }
      Dalli.logger.debug { "results from this server will be missing" }
      servers.delete(server)
    end
  end
  servers
end
ring() click to toggle source
# File lib/dalli/client.rb, line 309
def ring
  @ring ||= Dalli::Ring.new(
    @servers.map do |s|
     server_options = {}
      if s =~ %r{\Amemcached://}
        uri = URI.parse(s)
        server_options[:username] = uri.user
        server_options[:password] = uri.password
        s = "#{uri.host}:#{uri.port}"
      end
      Dalli::Server.new(s, @options.merge(server_options))
    end, @options
  )
end
validate_key(key) click to toggle source
# File lib/dalli/client.rb, line 342
def validate_key(key)
  raise ArgumentError, "key cannot be blank" if !key || key.length == 0
  key = key_with_namespace(key)
  if key.length > 250
    max_length_before_namespace = 212 - (namespace || '').size
    key = "#{key[0, max_length_before_namespace]}:md5:#{Digest::MD5.hexdigest(key)}"
  end
  return key
end