以前经常写 ruby 和 js, 对于以下的代码非常习惯:

item = {id: 1, name: 'Donald Clinton'}
attrs = [:id, :name]
attr_getters = attrs.map do |attr|
  lambda do
    "The #{attr} is #{item[attr]}"
  end
end
> attr_getters[0][]
"The id is 1"
> attr_getters[1][]
"The name is Donald Clinton"

var item = {id: 1, name: 'Donald Clinton'};
var attrs = ['id', 'name'];
var attr_getters = attrs.map(function (attr) {
  return function() { 
    var value_with_name = 'The ' + attr + ' is ' + item[attr];
    return value_with_name; 
  };
});
> attr_getters[0]()
'The id is 1'
> attr_getters[1]()
'The name is Donald Clinton'

最近的工作开始要写 python, 于是我想当然的也这么写了....

item = {'id': 1, 'name': 'Donald Clinton'}
attrs = ['id', 'name']

attr_getters = []
for attr in attrs:
    getter = lambda : 'The {0} is {1}'.format(attr, item[attr])
    attr_getters.append(getter)

可是并不 work!

>>> attr_getters[0]()
'The name is Donald Clinton'
>>> attr_getters[1]()
'The name is Donald Clinton'

仔细观察了一下代码... 这 for ... in ...: 虽然看起来很清真, 可是他并不是代码块. 我这傻乎乎的 lambda 的闭包始终绑定着 for 循环所在的环境, 于是当循环完成后, attr 就定格在了 'name' 上.

于是一个蠢呼呼的 fix 是在 lambda 外面再包一个

item = {'id': 1, 'name': 'Donald Clinton'}
attrs = ['id', 'name']

attr_getters = []
for attr in attrs:
    getter = (lambda a: lambda : 'The {0} is {1}'.format(a, item[a]))(attr)
    attr_getters.append(getter)

鉴于这很蠢...于是代码只能再改一下

item = {'id': 1, 'name': 'Donald Clinton'}
attrs = ['id', 'name']

def build_getter(attr):
    def getter():
        return 'The {0} is {1}'.format(attr, item[attr])
    return getter

attr_getters = map(build_getter, attrs)

lambda, 闭包等等, 这些东西在 python 世界里并不是一等公民, 随手定义函数的编程习惯在 python 中基本是完全不适用的. 单行 lambda 更是残废, 基本上想做点事情都必须预先定义一个函数才行, 没法定义匿名 block.

PS: 在 python 的世界里非常注重 callable 的. 小到一个函数, 大到一个类, 甚至一个实例, 只要定义了 __call__ 方法, 他就是 callable 的.

class A(object)
  def __init__(self, ...):
    ...

map(A, some_list_of_init_params) # => list of A

Even in MongoDB, sometimes your models may still have reference relations. Like ActiveRecord, the popular MongoDB ORM, mongoid, provides #includes method to perform eager loads.

Problems

However, it is a pain to specify them manually in codes. In most scenes, you just need the eager loading when you what to access the relation.

As described in Joel Turkel's post about Goldiloader, the eager loading should neither be a duty of controllers (why should they care about how the data is fetched) nor be one of views (they just present your data!).

Eager loading is more likely a job that the model it self should handle.

Better Practice

Just like Goldiloader for ActiveRecord, there can also be a plugin that makes mongoid do the eager loading automatically.

Say you have the following models for your blog site:

class User
  include Mongoid::Document
  has_one :device
  has_many :posts
end

class Post
  include Mongoid::Document
  belongs_to :user
  has_and_belongs_to_many :tags
end

class Tag
  include Mongoid::Document
  has_and_belongs_to_many :posts, fully_load: true
end

When you call

@user = User.find(id)
@posts = @user.posts.limit(10) # 

# access the tags
@posts.each do |post|
  puts post.title
  puts post.tags.map(&:name).join(', ')
end

Mongoid will perform 12 queries: 1 on users, 1 on posts and 10 on tags:

db.users.find('_id' : id);
db.posts.find('user_id' : user_id);
db.tags.find('post_ids' : post0_id);
db.tags.find('post_ids' : post1_id);
...
db.tags.find('post_ids' : post9_id);

But as an experienced developer, you could easily figure out that the eager loading on :tag relation should be performed. In fact, adding #includes to the criteria is a default action, manually done by the developers.

Why not make it a default to mongoid?

When you access the tag relation and is about to query it in your db, perform an eager loading on :tag just before the query:

# before query a relation
# the query loading method of a relation is
# Mongoid::Relations::Targets::Enumerable#unloaded_documents

def before_unloaded_documents
  owner = relation.base # get the relation owner
  metadata = relation.relation_metadata # get the relation metadata

  # DO THE EAGER LOADING
  # 
  #   Requires some trick to access the owners collection through the owner.
  #   Add an reference to the collection into model
end

Then when you run the previous @posts.each..., it will results to only 3 quires, 1 for each:

db.users.find('_id' : id);
db.posts.find('user_id' : user_id);
db.tags.find('post_ids' : {'$in' : post_ids});

Wow, you do not need to specify any eager loads. (The modified) Mongoid will do it automatically for you!

Conclusion

Eager loads are performance boosters, and should be an default behaviour in most situations. The developers should be aware of them, but not repeatly code .includes(:...) just because you need it everywhere.
I've written a gem mongoload to make your mongoid an automatic eager loader. Check it on GitHub.

ES6 ships with a very handy syntax sugar to build Object(aka Hash in Ruby).

var year = 2016;
var month = 3;
var day = 30;
var theDate = { year, month, day };

Can I use the syntax in ruby? calling something simple as HandyHash(:year, :month, :day)?

To get the variable values, the method #HandyHash must be aware of the context who calls him. This is not recommend in Ruby.
If you pass bindin to the method, the paramaters become (binding, :year, :month, :day)... Weird. Why I should call binding here.

Another approach to get the caller context binding is to pass a block defined in it. To get the hash keys, I decide to wrap an array in the block.

def HandyHash(&list_block)
  variables = yield.map(&:to_s)
  Hash[
    variables.map do |variable|
      [variable.to_sym, list_block.binding.local_variable_get(variable)]
    end
  ]
end

Suppose you have year, month, day = 2016, 3, 30. Then you can build the Hash with HandyHash{[:year, :month, :day]}.

I have wrapped the logic in to a gem called simple_hash (the Github repo is here).

最(好几个)近(月之前), 在公司前端同学的推荐下接触了GraphQL, 感觉一阵清爽. 如今使用了一段时间, 可以汇报一下使用感受.

GraphQL?

物如其名, GraphQL 是一种 Query Language, 由世界上最好语言的领军者 Facebook 公司发明. 它的出现是为了满足服务端向客户端灵活输出数据的需要.

Restful API 不行吗?

当然行. Restful 是包括我在内, 各位后端同学都非常熟悉的朋友. 从早年凭感觉写的jsp, asp们, 到后来做的 WebService API, 再到有哲学思想指导的 Restful API. 可以很明显的感觉到, 这些前后端的数据交互都是由后端在主导. 后端决定了数据格式, 决定了每个请求所能得到的数据.

可是! 时代不再是做个门户, 做个页面这么简单了. 如今更注重的是客户端侧的数据交互. 看看隔壁js们的各种前端框架, SPA, 数据绑定, MVVM 搞得热火朝天. 虽然后端依然承载着业务处理, 数据安全这样核心的功能. 但是客户端才是数据的需求方, 前端更清楚, 需要什么样的数据进行呈现, 使用什么样的数据进行交互.

这样一来写接口的的时候就会有这样的经历, 随着前端页面或是客户端的快速改版, 对于字段的需求是在不断变化, 于是一个 API 不断地 v1, v2, v3...vN. 最后也改烦了: 干脆把每个 model 的字段, 可以展示的都展示出去好了! 牺牲了服务器的带宽和客户的流量, 不过总算是不用再繁琐的更改接口定义了.

何不让客户端主导?

就好像 ElasticSearch 的接口里一样, 可以告诉服务端我要请求什么字段. 我这里有个叫 Uesr 的 model, 它一对多拥有许多的 Messages. 一个请求 Messages 列表的 Request 看起来就像下面这样:

GET /messages
{
  "since": "20XX-YY-ZZTAA:BB:CC",
  "fields": [
    "id",
    "title",
    "content",
    "created_at",
    {
      "user": [
        "id",
        "name",
        "account"
      ]
    }
  ]
}

其实也挺好看的对不对...

GraphQL 的出现, 就是定义了一套更为清晰更为灵活的标准, 来满足客户端对于数据灵活的需要.

In GraphQL

定义结构

GraphQL 的核心是两个概念. 一个是 Type, 他定义了数据的结构.

type Message {
  id: String
  title: String
  content: String
  user: User
}

type User {
  id: String
  name: String
  account: String
  messages: [Message]
}

这是 GraphQL Spec 中定义 Type 用的语言. (虽然实际上我也没见到任何一个 GraphQL 的库用到过这些语言, 包括 Facebook 自己官方提供的例子... 不过这种细节不用在意, 这就是为了方便沟通用的东西.)
在这个 Type 定义中, 定义了两个 Type, 在 {} 中间的定义的东西叫做 Field. 也就是 GraphQL 两个核心中的另一个. Type 和 Field 的定义形成了 GraphQL 的 Schema.

当然只有散户的话是没有入口的. GraphQL 的默认 Schema 自带了两个 Type 作为入口. 一个是 QueryRoot, 一个是 MutationRoot. 下文再慢慢讲.

我们可以在 QueryRoot 里把 Message 挂上去.

type QueryRoot {
  messages(since: Date): [Message]
}

[] 代表列表, () 里的是接受的参数.

执行查询

有了上面的 Schema 之后, 就能进行 query 了.

query messages(since: "20XX-YY-ZZTAA:BB:CC") {
  id
  title
  content
  user {
    id
    name
    account
  }
}

看起来比restful版本的可选字段请求清楚很多了. user 作为关联对象, 也很自然的出现在相同的结构中.

还有个 Mutation 呢?

上面提到过, GraphQL 的 Schema 最上级有两个 Root Type, 一个是 QueryRoot, 一个是 MutationRoot. Mutation, 根据 spec 中的定义, 是"会让服务端数据发生变化的 query", 也就是有副作用的 query .
比如定义一个发消息的 mutation

type MutationRoot {
  createMessage(title: String, content: String): Message
}

这里 createMessage 是一个 mutation, 它返回的数据 Type 为 Message.

mutation createMessage(title: "森的妖精 GraphQL", content: "blahblahblahblah") {
  id
}

这样就能创建一个 Message, 并且指定服务端返回创建后的 Message ID.

GraphQL 的基本用法就是上面这些了. 详细的还会有 Interface, Union 等东西, 主要是为了解决多态和复用.

总结?

在 GraphQL 中服务端的职责在于定义数据结构和准备数据, 进行业务处理. 而数据的获取, 展示则由更接近用户的客户端来主导. 对客户端和服务端都是很大的解放. 对于 Restful 在呈现端的不灵活是一个很大的补充.

不足

要说讨厌的地方, 那就是 Mutation 了. 根据 Facebook 所提供的 spec, 所有的 Mutation 都挂载在 MutationRoot 之下. 用我们 coding 所习惯的说法, 都是同一个类的全局方法... 感觉很难受.

于是我们在开发中, 前后端慢慢形成了一种默契. 在 Type 中定义对象的 Mutation...
类似这样

type Fish {
  id: String
  name: String
  Price: Float
  inStock: Int
  mutation: FishMutation
}

type FishMutation {
  update: UpdateFishMutation
  sell: SellFishMutation
}

于是就能

query {
  fish(id: "1") {
    id
    name
    price
    mutation {
      sell(weight: 100) {
        inStock
      }
    }
  }
}

这样看上去稍微舒服点...

Grape的params函数是非常方便用于定义API参数的工具. 这次做到一个接口要传入GeoJSON格式的参数. 与世界在接口上直接写了:

params do
  requires :location, type: GeoJSONPoint, desc: "地点"
end

class GeoJSONPoint
  attr_accessor :type, :coordinates
  
  def initialize(coordinates=[])
    @coordinates = coordinates
  end
  
  # always a point
  def type
    "Point"
  end
end

不过这样显然是不行的, 一定会出400 Bad Request. Grape毕竟不知道GeoJSONPoint这个类是怎么转换的. 读了读Grape的代码, 发现Grape是用 Virtus 这个gem来转换参数=>对象的.
于是这个GeoJSONPoint的类就需要用Virtus的方式来定义. 于是代码变成了

class GeoJSONPoint
  include Virtus.model
  
  attribute :type, String
  attribute :coordinates, Array
  
  def initialize(coordinates=[])
    @coordinates = coordinates
  end
  
  # always a point
  def type
    "Point"
  end
end

先用Virtus的方法试一下, 结果OK:

coercer = Virtus::Attribute.build(GeoJSONPoint)
point = coercer.coerce({type: "Point", coordinates: [190, 50]})
=> #<GeoJSONPoint:0x007ffdf3a61618 @coordinates={:type=>"Point", :coordinates=>[190, 50]}>

再到RestClient里发个请求看看:

{"location" : { 
  "type" : "Point", 
  "coordinates" : [190, 50] 
}}

结果:

{
  "application":{"name":"TestServer","status":"ok"},
  "request":{
    "headers":{
     "Version":"HTTP/1.1",
     "Host":"localhost:3000",
     "Connection":"Keep-Alive",
     "User-Agent":"Apache-HttpClient/4.3.5 (java 1.5)",
     "Accept-Encoding":"gzip,deflate"
   },
   "params":{
     "location":{
       "type":"Point",
       "coordinates":[190,50]}}},
 "env":"127.0.0.1"
}

Grape能够成功把GeoJSONPoint转换出来.

最近手头有一个需要快速开发的新Web项目, 于是想了想还是不用sinatra, 直接用RoR来写. 因为需要为手机提供API, 感觉在Rails上面架设sinatra有点自欺欺人的感觉, 于是第一次尝试在Rails上面加一层Grape... 于是问题就来了.

Sorcery在Session/Cookies方面的操作完全依赖Rails的API, 以前用sinatra的时候, 基本上没有感觉到什么不适, 这次在Grape上使用Sorcery就爆炸了OTL. Grape::Endpoint.send(:include, Sorcery::Controller)之后就开始各种undefined. 看了看大概都集中在ActionDispatch和ActionController里. 下午大概的做了个hotfix, 可惜因为Grape自用的Cookies和ActionDispatch::Cookies区别实在太大, Grape的remember_me功能在grape上暂时用不了.

module API
  module SorceryAdapter

    AUTHENTICITY_TOKEN_LENGTH = 32

    def self.included(mod)
      mod.instance_eval {
        helpers do
          ### Adapt for Sorcery (some directly taken from rails codes)

          # Get session
          def session
            env[Rack::Session::Abstract::ENV_SESSION_KEY]
          end

          # Disable remember_me because of the cookies type conflict
          def current_user
            unless defined?(@current_user)
              @current_user = login_from_session || nil
            end
            @current_user
          end

          ## ActionDispatch::Request
          def reset_session
            if session && session.respond_to?(:destroy)
              session.destroy
            else
              self.session = {}
            end
            @env['action_dispatch.request.flash_hash'] = nil
          end

          ## ActionController::RequestForgeryProtection::ProtectionMethods::NullSession
          # Sets the token value for the current session.
          def form_authenticity_token
            masked_authenticity_token(session)
          end

          def real_csrf_token(session)
            session[:_csrf_token] ||= SecureRandom.base64(AUTHENTICITY_TOKEN_LENGTH)
            Base64.strict_decode64(session[:_csrf_token])
          end

          # Creates a masked version of the authenticity token that varies
          # on each request. The masking is used to mitigate SSL attacks
          # like BREACH.
          def masked_authenticity_token(session)
            one_time_pad = SecureRandom.random_bytes(AUTHENTICITY_TOKEN_LENGTH)
            encrypted_csrf_token = xor_byte_strings(one_time_pad, real_csrf_token(session))
            masked_token = one_time_pad + encrypted_csrf_token
            Base64.strict_encode64(masked_token)
          end

          def xor_byte_strings(s1, s2)
            s1.bytes.zip(s2.bytes).map { |(c1,c2)| c1 ^ c2 }.pack('c*')
          end
        end
      }
    end
  end
end

想着把Blog的backend db从sqlite换到mariadb, 结果不小心把.db文件删掉了... 明明一直在跟服务器打交道, 结果大意地把数据库删了>< 好糟糕otl... 总之钻石墓场就这样了OTL...