<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>「浮生若梦」 - sczyh30&#39;s blog</title>
  <subtitle>踏歌长行，梦想永在。</subtitle>
  <link href="/atom.xml" rel="self"/>
  
  <link href="http://www.sczyh30.com/"/>
  <updated>2018-09-23T14:18:08.000Z</updated>
  <id>http://www.sczyh30.com/</id>
  
  <author>
    <name>sczyh30</name>
    
  </author>
  
  <generator uri="http://hexo.io/">Hexo</generator>
  
  <entry>
    <title>Hystrix 1.5 滑动窗口实现原理总结</title>
    <link href="http://www.sczyh30.com/posts/%E9%AB%98%E5%8F%AF%E7%94%A8%E6%9E%B6%E6%9E%84/netflix-hystrix-1-5-sliding-window/"/>
    <id>http://www.sczyh30.com/posts/高可用架构/netflix-hystrix-1-5-sliding-window/</id>
    <published>2017-12-03T16:00:00.000Z</published>
    <updated>2018-09-23T14:18:08.000Z</updated>
    
    <content type="html"><![CDATA[<h1 id="总览">总览</h1><p>Netflix Hystrix 通过类似滑动窗口的数据结构来统计调用的指标数据。Hystrix 1.5 将滑动窗口设计成了数据流（reactive stream, RxJava 中的 <code>Observable</code>）的形式。通过消费数据流的形式利用滑动窗口，并对数据流进行变换后进行后续的操作，可以让开发者更加灵活地去使用。由于 Hystrix 里大量使用了 RxJava，再加上滑动窗口本质就是不断变换的数据流，滑动窗口中每个桶的数据都来自于源源不断的事件，因此滑动窗口非常适合用观察者模式和响应式编程思想的 RxJava 实现。使用 RxJava 实现有一大好处：可以通过 RxJava 的一系列操作符来实现滑动窗口，从而可以依赖 RxJava 的线程模型来保证数据写入和聚合的线程安全，将这一系列的机制交给 RxJava。所有的操作都是在 RxJava 的后台线程上进行的，RxJava 会保证操作的有序性和线程安全（参见 <a href="http://reactivex.io/documentation/contract.html" target="_blank" rel="external">The Observable Contract</a>）。</p>
<p>这里我们就以 Hystrix 熔断器依赖的记录调用情况统计的 <code>HealthCountsStream</code> 为例来看一下 Hystrix 1.5 是如何利用 RxJava 将滑动窗口抽象并实现成 reactive stream 的，以及如何去消费对应的数据流。</p>
<p>滑动窗口的实现都位于 <code>com.netflix.hystrix.metric.consumer</code> 包下，这里只挑 <code>BucketedRollingCounterStream</code> 这条线的实现来分析。首先先看一下类的继承结构：</p>
<p><img src="/assets/blog-img-k1/Hystrix-BucketedRollingCounterStream-hierarchy.png" alt="Class hierarchy of BucketedRollingCounterStream"></p>
<p>最顶层的 <code>BucketedCounterStream</code> 抽象类提供了基本的桶计数器实现，按配置的时间间隔将所有事件聚合成桶；<code>BucketedRollingCounterStream</code> 抽象类在其基础上实现滑动窗口，并聚合成指标数据；而最底下一层的类则是各种具体的实现，比如 <code>HealthCountsStream</code> 最终会聚合成健康检查数据（<code>HystrixCommandMetrics.HealthCounts</code>，统计调用成功和失败的计数），供 <code>HystrixCircuitBreaker</code> 使用。</p>
<h1 id="BucketedCounterStream">BucketedCounterStream</h1><h2 id="总览-1">总览</h2><p><code>BucketedCounterStream</code> 抽象类提供了基本的桶计数器实现。用户在使用 Hystrix 的时候一般都要配两个值：<code>timeInMilliseconds</code> 和 <code>numBuckets</code>，前者代表滑动窗口的长度（时间间隔），后者代表滑动窗口中桶的个数，那么每个桶对应的窗口长度就是 <code>bucketSizeInMs = timeInMilliseconds / numBuckets</code>（记为一个单元窗口周期）。<code>BucketedCounterStream</code> 每隔一个单元窗口周期（<code>bucketSizeInMs</code>）就把这段时间内的所有调用事件聚合到一个桶内。我们来看一下它的实现，首先来看一下它的泛型定义：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="keyword">abstract</span> <span class="class"><span class="keyword">class</span> <span class="title">BucketedCounterStream</span>&lt;<span class="title">Event</span> <span class="keyword">extends</span> <span class="title">HystrixEvent</span>, <span class="title">Bucket</span>, <span class="title">Output</span>&gt; </span>&#123;</div><div class="line">    <span class="comment">// ...</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>BucketedCounterStream</code> 的泛型里接受三个类型参数，其中第一个 <code>Event</code> 类型代表 Hystrix 中的调用事件，如命令开始执行、命令执行完成等。这种事件驱动的设计也非常符合 RxJava 的思想，每个调用者都向订阅者发布事件，订阅者将事件聚合成调用指标；第二个 <code>Bucket</code> 类型代表桶的类型，第三个 <code>Output</code> 类型代表数据聚合的最终输出类型。</p>
<p><code>BucketedCounterStream</code> 核心代码在构造函数里（为了可读性起见，将所有可以用 lambda expression 的地方都转换成了 lambda expression，下同）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> <span class="keyword">final</span> <span class="keyword">int</span> numBuckets;</div><div class="line"><span class="keyword">protected</span> <span class="keyword">final</span> Observable&lt;Bucket&gt; bucketedStream;</div><div class="line"></div><div class="line"><span class="keyword">private</span> <span class="keyword">final</span> Func1&lt;Observable&lt;Event&gt;, Observable&lt;Bucket&gt;&gt; reduceBucketToSummary;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">protected</span> <span class="title">BucketedCounterStream</span><span class="params">(<span class="keyword">final</span> HystrixEventStream&lt;Event&gt; inputEventStream, <span class="keyword">final</span> <span class="keyword">int</span> numBuckets, <span class="keyword">final</span> <span class="keyword">int</span> bucketSizeInMs,</span></span></div><div class="line">                                <span class="keyword">final</span> Func2&lt;Bucket, Event, Bucket&gt; appendRawEventToBucket) &#123;</div><div class="line">    <span class="keyword">this</span>.numBuckets = numBuckets;</div><div class="line">    <span class="keyword">this</span>.reduceBucketToSummary = eventBucket -&gt; eventBucket.reduce(getEmptyBucketSummary(), appendRawEventToBucket);</div><div class="line"></div><div class="line">    <span class="keyword">final</span> List&lt;Bucket&gt; emptyEventCountsToStart = <span class="keyword">new</span> ArrayList&lt;&gt;();</div><div class="line">    <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; numBuckets; i++) &#123;</div><div class="line">        emptyEventCountsToStart.add(getEmptyBucketSummary());</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="keyword">this</span>.bucketedStream = Observable.defer(() -&gt; &#123;</div><div class="line">        <span class="keyword">return</span> inputEventStream</div><div class="line">                .observe()</div><div class="line">                .window(bucketSizeInMs, TimeUnit.MILLISECONDS) <span class="comment">//bucket it by the counter window so we can emit to the next operator in time chunks, not on every OnNext</span></div><div class="line">                .flatMap(reduceBucketToSummary)                <span class="comment">//for a given bucket, turn it into a long array containing counts of event types</span></div><div class="line">                .startWith(emptyEventCountsToStart);           <span class="comment">//start it with empty arrays to make consumer logic as generic as possible (windows are always full)</span></div><div class="line">    &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>其中 <code>bucketedStream</code> 即为本次得到的数据流（类型为 RxJava 中的 <code>Observable</code>，即观察者模式中的 Publisher，会源源不断地产生事件/数据），里面最核心的逻辑就是如何将一个一个的事件按一段时间聚合成一个桶。我们可以看到 <code>bucketedStream</code> 是经事件源 <code>inputEventStream</code> 变换而成的，事件源的类型为 <code>HystrixEventStream&lt;Event&gt;</code>，它代表事件流接口：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">HystrixEventStream</span>&lt;<span class="title">E</span> <span class="keyword">extends</span> <span class="title">HystrixEvent</span>&gt; </span>&#123;</div><div class="line"></div><div class="line">    <span class="function">Observable&lt;E&gt; <span class="title">observe</span><span class="params">()</span></span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>其中 <code>observe</code> 方法返回这个事件流对应的发布者 <code>Observable</code>，订阅者可以对事件进行变换并消费。</p>
<h2 id="事件流的写入">事件流的写入</h2><p>Hystrix 中执行函数以命令模式封装成了一个一个命令（<code>Command</code>），每个命令执行时都会触发某个事件，其中命令执行完成事件（<code>HystrixCommandCompletion</code>）是 Hystrix 中最核心的事件，它可以代表某个命令执行成功、超时、异常等等的各种的状态，与服务调用的熔断息息相关。熔断器的计数依赖于 <code>HystrixCommandCompletion</code> 事件，因此这里我们只关注这个事件对应的事件流，其余类型的事件流原理类似。</p>
<p>那么这个事件流中的事件是从哪里发布的呢？我们来看一下相关的具体实现 - <code>HystrixCommandCompletionStream</code>（仅核心代码）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">HystrixCommandCompletionStream</span> <span class="keyword">implements</span> <span class="title">HystrixEventStream</span>&lt;<span class="title">HystrixCommandCompletion</span>&gt; </span>&#123;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> HystrixCommandKey commandKey;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> Subject&lt;HystrixCommandCompletion, HystrixCommandCompletion&gt; writeOnlySubject;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> Observable&lt;HystrixCommandCompletion&gt; readOnlyStream;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> ConcurrentMap&lt;String, HystrixCommandCompletionStream&gt; streams = <span class="keyword">new</span> ConcurrentHashMap&lt;String, HystrixCommandCompletionStream&gt;();</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> HystrixCommandCompletionStream <span class="title">getInstance</span><span class="params">(HystrixCommandKey commandKey)</span> </span>&#123;</div><div class="line">        <span class="comment">// 此段代码略，大致逻辑为：若对应的 CommandKey 的事件流已创建就从缓存中取出，否则就新创建并缓存起来，保证每个 CommandKey 只有一个实例</span></div><div class="line">    &#125;</div><div class="line"></div><div class="line">    HystrixCommandCompletionStream(<span class="keyword">final</span> HystrixCommandKey commandKey) &#123;</div><div class="line">        <span class="keyword">this</span>.commandKey = commandKey;</div><div class="line"></div><div class="line">        <span class="keyword">this</span>.writeOnlySubject = <span class="keyword">new</span> SerializedSubject&lt;HystrixCommandCompletion, HystrixCommandCompletion&gt;(PublishSubject.&lt;HystrixCommandCompletion&gt;create());</div><div class="line">        <span class="keyword">this</span>.readOnlyStream = writeOnlySubject.share();</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">reset</span><span class="params">()</span> </span>&#123;</div><div class="line">        streams.clear();</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">write</span><span class="params">(HystrixCommandCompletion event)</span> </span>&#123;</div><div class="line">        writeOnlySubject.onNext(event);</div><div class="line">    &#125;</div><div class="line"></div><div class="line"></div><div class="line">    <span class="meta">@Override</span></div><div class="line">    <span class="function"><span class="keyword">public</span> Observable&lt;HystrixCommandCompletion&gt; <span class="title">observe</span><span class="params">()</span> </span>&#123;</div><div class="line">        <span class="keyword">return</span> readOnlyStream;</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>从代码里我们可以看到 <code>write</code> 方法里通过向某个 <code>Subject</code> 发布事件来实现了发布的逻辑，那么 <code>Subject</code> 又是什么呢？简单来说，<a href="http://reactivex.io/documentation/subject.html" target="_blank" rel="external"><code>Subject</code></a> 就像是一个桥梁，既可以作为发布者 <code>Observable</code>，又可以作为订阅者 <code>Observer</code>。它可以作为发布者和订阅者之间的一个“代理”，提供额外的功能（如流量控制、缓存等）。这里的 <code>writeOnlySubject</code> 是经过 <code>SerializedSubject</code> 封装的 <code>PublishSubject</code>。<code>PublishSubject</code> 可以看做 hot observable。为了保证调用的顺序（根据 <a href="http://reactivex.io/documentation/contract.html" target="_blank" rel="external">The Observable Contract</a>，每个事件的产生需要满足顺序上的偏序关系，即使是在不同线程产生），需要用 <code>SerializedSubject</code> 封装一层来保证事件真正地串行地产生。这里还有一个问题，就是不同的发布者调用 <code>write</code> 方法发布事件时，线程上下文可能都不同，那么如何保证其线程安全呢？Hystrix 1.5 通过使用 ThreadLocal 来保证每个线程都有一份 <code>Subject</code> 的实例，确保事件发布的线程安全。相关代码位于 <code>HystrixThreadEventStream</code> 内（已略去其它事件的代码）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">HystrixThreadEventStream</span> </span>&#123;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> threadId;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> String threadName;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> Subject&lt;HystrixCommandExecutionStarted, HystrixCommandExecutionStarted&gt; writeOnlyCommandStartSubject;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> Subject&lt;HystrixCommandCompletion, HystrixCommandCompletion&gt; writeOnlyCommandCompletionSubject;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> Subject&lt;HystrixCollapserEvent, HystrixCollapserEvent&gt; writeOnlyCollapserSubject;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> ThreadLocal&lt;HystrixThreadEventStream&gt; threadLocalStreams = <span class="keyword">new</span> ThreadLocal&lt;HystrixThreadEventStream&gt;() &#123;</div><div class="line">        <span class="meta">@Override</span></div><div class="line">        <span class="function"><span class="keyword">protected</span> HystrixThreadEventStream <span class="title">initialValue</span><span class="params">()</span> </span>&#123;</div><div class="line">            <span class="keyword">return</span> <span class="keyword">new</span> HystrixThreadEventStream(Thread.currentThread());</div><div class="line">        &#125;</div><div class="line">    &#125;;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> Action1&lt;HystrixCommandCompletion&gt; writeCommandCompletionsToShardedStreams = commandCompletion -&gt; &#123;</div><div class="line">        HystrixCommandCompletionStream commandStream = HystrixCommandCompletionStream.getInstance(commandCompletion.getCommandKey());</div><div class="line">        commandStream.write(commandCompletion);</div><div class="line"></div><div class="line">        <span class="keyword">if</span> (commandCompletion.isExecutedInThread() || commandCompletion.isResponseThreadPoolRejected()) &#123;</div><div class="line">            HystrixThreadPoolCompletionStream threadPoolStream = HystrixThreadPoolCompletionStream.getInstance(commandCompletion.getThreadPoolKey());</div><div class="line">            threadPoolStream.write(commandCompletion);</div><div class="line">        &#125;</div><div class="line">    &#125;;</div><div class="line"></div><div class="line">    <span class="comment">/* package */</span> HystrixThreadEventStream(Thread thread) &#123;</div><div class="line">        <span class="keyword">this</span>.threadId = thread.getId();</div><div class="line">        <span class="keyword">this</span>.threadName = thread.getName();</div><div class="line">        writeOnlyCommandCompletionSubject = PublishSubject.create();</div><div class="line"></div><div class="line">        writeOnlyCommandCompletionSubject</div><div class="line">                .onBackpressureBuffer()</div><div class="line">                .doOnNext(writeCommandCompletionsToShardedStreams)</div><div class="line">                .unsafeSubscribe(Subscribers.empty());</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> HystrixThreadEventStream <span class="title">getInstance</span><span class="params">()</span> </span>&#123;</div><div class="line">        <span class="keyword">return</span> threadLocalStreams.get();</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">shutdown</span><span class="params">()</span> </span>&#123;</div><div class="line">        writeOnlyCommandStartSubject.onCompleted();</div><div class="line">        writeOnlyCommandCompletionSubject.onCompleted();</div><div class="line">        writeOnlyCollapserSubject.onCompleted();</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="comment">// 执行完毕/异常/超时都会调用此方法</span></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">executionDone</span><span class="params">(ExecutionResult executionResult, HystrixCommandKey commandKey, HystrixThreadPoolKey threadPoolKey)</span> </span>&#123;</div><div class="line">        HystrixCommandCompletion event = HystrixCommandCompletion.from(executionResult, commandKey, threadPoolKey);</div><div class="line">        writeOnlyCommandCompletionSubject.onNext(event);</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里 Hystrix 通过 ThreadLocal 为每个不同的线程都创建了不同的 <code>HystrixThreadEventStream</code>，里面的 <code>Subject</code> 都是 write-only, thread-safe 的。Hystrix 在这里额外加了一层 <code>writeOnlyCommandCompletionSubject</code>，提供额外的流量控制机制（<code>onBackpressureBuffer</code>），消费者太慢时这里会积压。其中会调用 <code>HystrixCommandCompletionStream</code> 的 <code>write</code> 方法产生对应的事件。</p>
<p><code>executionDone</code> 方法最后会经 <code>HystrixCommandMetrics</code> 类的 <code>markCommandDone</code> 方法进行调用。<code>HystrixCommandMetrics</code> 是 Hystrix 中另一个重要的类，从中可以获取各种指标数据的流的实例。最后 Hystrix 会在对应命令执行完毕后，调用 <code>markCommandDone</code> 进行数据记录，并发布对应的事件。相关代码位于 <a href="https://github.com/Netflix/Hystrix/blob/master/hystrix-core/src/main/java/com/netflix/hystrix/AbstractCommand.java#L947" target="_blank" rel="external"><code>AbstractCommand</code></a> 类内：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleCommandEnd</span><span class="params">(<span class="keyword">boolean</span> commandExecutionStarted)</span> </span>&#123;</div><div class="line">    Reference&lt;TimerListener&gt; tl = timeoutTimer.get();</div><div class="line">    <span class="keyword">if</span> (tl != <span class="keyword">null</span>) &#123;</div><div class="line">        tl.clear();</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="keyword">long</span> userThreadLatency = System.currentTimeMillis() - commandStartTimestamp;</div><div class="line">    executionResult = executionResult.markUserThreadCompletion((<span class="keyword">int</span>) userThreadLatency);</div><div class="line">    <span class="keyword">if</span> (executionResultAtTimeOfCancellation == <span class="keyword">null</span>) &#123;</div><div class="line">        metrics.markCommandDone(executionResult, commandKey, threadPoolKey, commandExecutionStarted);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">        metrics.markCommandDone(executionResultAtTimeOfCancellation, commandKey, threadPoolKey, commandExecutionStarted);</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="keyword">if</span> (endCurrentThreadExecutingCommand != <span class="keyword">null</span>) &#123;</div><div class="line">        endCurrentThreadExecutingCommand.call();</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>AbstractCommand</code> 类是 Hystrix 命令模式执行模型的实现，整合了资源隔离、熔断器等各种高可用机制，是整个 Hystrix 的核心。</p>
<h2 id="共享的事件流">共享的事件流</h2><p>上面我们探究了事件流的发布原理，以及如何保证写的线程安全。那么事件流写入到 <code>writeOnlySubject</code> 以后，如何被订阅者消费呢？如何保证多个订阅者都可以对事件流进行消费，并且序列一致呢？我们回到之前的 <code>observe</code> 方法，<code>observe</code> 方法返回的是一个 <code>readOnlyStream</code>：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> Observable&lt;HystrixCommandCompletion&gt; <span class="title">observe</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> readOnlyStream;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>readOnlyStream</code> 是 <code>writeOnlySubject</code> 的只读版本，它是通过 <code>share</code> 操作符产生的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">this</span>.readOnlyStream = writeOnlySubject.share();</div></pre></td></tr></table></figure>
<p>Hystrix 通过 RxJava 的 <a href="http://reactivex.io/documentation/operators/refcount.html" target="_blank" rel="external"><code>share</code></a> 操作符产生一种特殊的 <code>Observable</code>：当有一个订阅者去消费事件流时它就开始产生事件，可以有多个订阅者去订阅，同一时刻收到的事件是一致的；直到最后一个订阅者取消订阅以后，事件流才停止产生事件。其底层实现非常有意思：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">final</span> Observable&lt;T&gt; <span class="title">share</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> publish().refCount();</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在执行 <code>publish</code> 的时候，<code>Observable</code> 会被变换成为一个 <code>ConnectableObservable</code>。这种 <code>ConnectableObservable</code> 只会在进行连接操作（<code>connect</code>）以后才会产生数据（连接后行为类似于 hot observable）。而 <code>share</code> 操作底层的 <code>refCount</code> 操作符就帮我们做了这样的操作：<code>refCount</code> 底层维护着一个引用计数器，代表绑定的订阅者数目。当第一个订阅者去消费事件流的时候，引用计数大于 0，<code>refCount</code> 底层会自动进行 <code>connect</code>，从而触发事件流产生事件；当最后一个订阅者取消订阅以后，引用计数归零，<code>refCount</code> 底层就会自动进行 <code>disconnect</code>，事件流停止产生事件。也就是说，这样的一个可以被多个订阅者共享的事件流，底层是基于引用计数法来管理事件的产生的，和智能指针的思想类似。</p>
<p><img src="http://reactivex.io/documentation/operators/images/publishRefCount.png" alt="Reactive stream of refCount"></p>
<h2 id="事件流聚合为桶">事件流聚合为桶</h2><p>上面我们研究完了事件流是如何产生的，接下来就回归到事件流聚合为桶的逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">this</span>.bucketedStream = Observable.defer(() -&gt; &#123; <span class="comment">// defer 的意思是 lazy 创建</span></div><div class="line">    <span class="keyword">return</span> inputEventStream</div><div class="line">            .observe()</div><div class="line">            .window(bucketSizeInMs, TimeUnit.MILLISECONDS) <span class="comment">// 按单元窗口长度来将某个时间段内的调用事件聚集起来</span></div><div class="line">            .flatMap(reduceBucketToSummary)                <span class="comment">// 将每个单元窗口内聚集起来的事件集合聚合成桶</span></div><div class="line">            .startWith(emptyEventCountsToStart);           <span class="comment">// 为了保证窗口的完整性，开始的时候先产生一串空的桶</span></div><div class="line">&#125;);</div></pre></td></tr></table></figure>
<p>其中的核心是 <a href="http://reactivex.io/documentation/operators/window.html" target="_blank" rel="external"><code>window</code></a> 操作符，它可以按单元窗口长度来将某个时间段内的调用事件聚集起来，此时数据流里每个对象都是一个集合：<code>Observable&lt;Event&gt;</code>，所以需要将其聚集成桶类型以将其扁平化。Hystrix 通过 RxJava 的 <code>reduce</code> 操作符进行“归纳”操作，将一串事件归纳成一个桶：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">this</span>.reduceBucketToSummary = eventBucket -&gt; eventBucket.reduce(getEmptyBucketSummary(), appendRawEventToBucket);</div></pre></td></tr></table></figure>
<p>其中我们需要提供桶的初值（即空桶），并要提供聚合函数来进行聚合，类型为 <code>Bucket -&gt; Event -&gt; Bucket</code>（代表对于每个 <code>Event</code>，都将其聚合到 <code>Bucket</code> 中，并返回聚合后的 <code>Bucket</code>）。不同的实现对应的 <code>Bucket</code> 和规约函数不同，比如熔断器依赖的 <code>HealthCountsStream</code> 就以 <code>long[]</code> 来作为每个桶。</p>
<blockquote>
<p>注：此处的 <code>window(timespan, unit)</code> 操作符属于计算型操作符，默认会在 <code>Schedulers.computation()</code> 调度器下执行（CPU 密集型），其底层本质是线程数为 CPU 核数的线程池。RxJava 会确保其线程安全。</p>
</blockquote>
<h1 id="BucketedRollingCounterStream">BucketedRollingCounterStream</h1><p><code>BucketedRollingCounterStream</code> 按照滑动窗口的大小对每个单元窗口产生的桶进行聚合，这也是 Hystrix 1.5 中滑动窗口的抽象实现。其核心实现仍然位于构造函数内：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="keyword">abstract</span> <span class="class"><span class="keyword">class</span> <span class="title">BucketedRollingCounterStream</span>&lt;<span class="title">Event</span> <span class="keyword">extends</span> <span class="title">HystrixEvent</span>, <span class="title">Bucket</span>, <span class="title">Output</span>&gt; <span class="keyword">extends</span> <span class="title">BucketedCounterStream</span>&lt;<span class="title">Event</span>, <span class="title">Bucket</span>, <span class="title">Output</span>&gt; </span>&#123;</div><div class="line">    <span class="keyword">private</span> Observable&lt;Output&gt; sourceStream;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> AtomicBoolean isSourceCurrentlySubscribed = <span class="keyword">new</span> AtomicBoolean(<span class="keyword">false</span>);</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">protected</span> <span class="title">BucketedRollingCounterStream</span><span class="params">(HystrixEventStream&lt;Event&gt; stream, <span class="keyword">final</span> <span class="keyword">int</span> numBuckets, <span class="keyword">int</span> bucketSizeInMs,</span></span></div><div class="line">                                           <span class="keyword">final</span> Func2&lt;Bucket, Event, Bucket&gt; appendRawEventToBucket,</div><div class="line">                                           <span class="keyword">final</span> Func2&lt;Output, Bucket, Output&gt; reduceBucket) &#123;</div><div class="line">        <span class="keyword">super</span>(stream, numBuckets, bucketSizeInMs, appendRawEventToBucket);</div><div class="line">        Func1&lt;Observable&lt;Bucket&gt;, Observable&lt;Output&gt;&gt; reduceWindowToSummary = window -&gt; window.scan(getEmptyOutputValue(), reduceBucket).skip(numBuckets);</div><div class="line">        <span class="keyword">this</span>.sourceStream = bucketedStream      <span class="comment">// 数据流，每个对象代表单元窗口产生的桶</span></div><div class="line">                .window(numBuckets, <span class="number">1</span>)          <span class="comment">// 按照滑动窗口桶的个数进行桶的聚集</span></div><div class="line">                .flatMap(reduceWindowToSummary) <span class="comment">// 将一系列的桶聚集成最后的数据对象</span></div><div class="line">                .doOnSubscribe(() -&gt; isSourceCurrentlySubscribed.set(<span class="keyword">true</span>))</div><div class="line">                .doOnUnsubscribe(() -&gt; isSourceCurrentlySubscribed.set(<span class="keyword">false</span>))</div><div class="line">                .share()                        <span class="comment">// 不同的订阅者看到的数据是一致的</span></div><div class="line">                .onBackpressureDrop();          <span class="comment">// 流量控制，当消费者消费速度过慢时就丢弃数据，不进行积压</span></div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="meta">@Override</span></div><div class="line">    <span class="function"><span class="keyword">public</span> Observable&lt;Output&gt; <span class="title">observe</span><span class="params">()</span> </span>&#123;</div><div class="line">        <span class="keyword">return</span> sourceStream;</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="comment">/* package-private */</span> <span class="function"><span class="keyword">boolean</span> <span class="title">isSourceCurrentlySubscribed</span><span class="params">()</span> </span>&#123;</div><div class="line">        <span class="keyword">return</span> isSourceCurrentlySubscribed.get();</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>构造函数后两个参数参数分别代表两个函数：将事件流聚合成桶的函数（<code>appendRawEventToBucket</code>） 以及 将桶聚合成输出对象的函数（<code>reduceBucket</code>）。</p>
<p>我们看到 <code>BucketedRollingCounterStream</code> 实现了 <code>observe</code> 方法，返回了一个 <code>Observable&lt;Output&gt;</code> 类型的发布者 <code>sourceStream</code>，供订阅者去消费。这里的 <code>sourceStream</code> 应该就是滑动窗口的终极形态了，那么它是如何变换得到的呢？这里面的核心还是 <code>window</code> 和 <code>flatMap</code> 算子。这里的 <code>window</code> 算子和之前的版本不同，它可以将数据流中的一定数量的数据聚集成一个集合，它的第二个参数 <code>skip=1</code> 的意思就是按照步长为 1 在数据流中滑动，不断聚集对象，这即为滑动窗口的真正实现。到这里每个窗口都已经形成了，下一步就是要对窗口进行聚合了。注意这里聚合操作没有用 <code>reduce</code>，而是用了 <code>scan</code> + <code>skip(numBuckets)</code> 的组合：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">Func1&lt;Observable&lt;Bucket&gt;, Observable&lt;Output&gt;&gt; reduceWindowToSummary = window -&gt; window.scan(getEmptyOutputValue(), reduceBucket).skip(numBuckets);</div></pre></td></tr></table></figure>
<p>这里每个集合的大小都是 <code>numBuckets</code>，看起来用 <code>reduce</code> 和 <code>scan</code> + <code>skip(numBuckets)</code> 没有什么区别，但是注意当数据流终结时，最后面的窗口大小都不满 <code>numBuckets</code>，这时候就需要把这些不完整的窗口给过滤掉来确保数据不缺失。这个地方也是开发的时候容易忽略的地方，很值得思考。</p>
<p>聚合完毕以后，基本的滑动窗口数据就OK了，为了支持多订阅者，还要进行 <code>share</code>；并且利用 <code>onBackpressureDrop</code> 操作符实现流量控制，此处当消费者跟不上的时候就直接丢掉数据，不进行积压。</p>
<p><img src="https://github.com/Netflix/Hystrix/wiki/images/rolling-stats-640.png" alt="Rolling status"></p>
<h1 id="HealthCountsStream">HealthCountsStream</h1><p>前面滑动窗口的抽象实现都已经分析完了，现在我们就来看一下其中的一个具体实现 - <code>HealthCountsStream</code>，它提供实时的健康检查数据（<code>HystrixCommandMetrics.HealthCounts</code>，统计调用成功和失败的计数）。</p>
<p>之前我们提到 <code>BucketedRollingCounterStream</code> 里面有三个类型参数和两个重要函数参数。<code>HealthCountsStream</code> 对应的三个类型参数分别为：</p>
<ul>
<li><code>Event</code>: <code>HystrixCommandCompletion</code>，代表命令执行完成。可以从中获取执行结果，并从中提取所有产生的事件（<code>HystrixEventType</code>）</li>
<li><code>Bucket</code>: 桶的类型为 <code>long[]</code>，里面统计了各种事件的个数。其中 index 为事件类型枚举对应的索引（<code>ordinal</code>），值为对应事件的个数</li>
<li><code>Output</code>: <code>HystrixCommandMetrics.HealthCounts</code>，里面统计了总的执行次数、失败次数以及失败百分比，供熔断器使用</li>
</ul>
<p>滑动窗口里用于将事件聚合成桶的函数实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> Func2&lt;<span class="keyword">long</span>[], HystrixCommandCompletion, <span class="keyword">long</span>[]&gt; appendEventToBucket = (initialCountArray, execution) -&gt; &#123;</div><div class="line">    ExecutionResult.EventCounts eventCounts = execution.getEventCounts();</div><div class="line">    <span class="keyword">for</span> (HystrixEventType eventType: ALL_EVENT_TYPES) &#123;</div><div class="line">        <span class="keyword">switch</span> (eventType) &#123;</div><div class="line">            <span class="keyword">case</span> EXCEPTION_THROWN: <span class="keyword">break</span>; <span class="comment">//this is just a sum of other anyway - don't do the work here</span></div><div class="line">            <span class="keyword">default</span>:</div><div class="line">                initialCountArray[eventType.ordinal()] += eventCounts.getCount(eventType);</div><div class="line">                <span class="keyword">break</span>;</div><div class="line">        &#125;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">return</span> initialCountArray;</div><div class="line">&#125;;</div></pre></td></tr></table></figure>
<p>滑动窗口里用于将每个窗口聚合成最终的统计数据的的函数实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> Func2&lt;HystrixCommandMetrics.HealthCounts, <span class="keyword">long</span>[], HystrixCommandMetrics.HealthCounts&gt; healthCheckAccumulator = HystrixCommandMetrics.HealthCounts::plus;</div><div class="line"></div><div class="line"><span class="comment">// 具体的实现，位于 HystrixCommandMetrics.HealthCounts 类内</span></div><div class="line"><span class="function"><span class="keyword">public</span> HealthCounts <span class="title">plus</span><span class="params">(<span class="keyword">long</span>[] eventTypeCounts)</span> </span>&#123;</div><div class="line">    <span class="keyword">long</span> updatedTotalCount = totalCount; <span class="comment">// 之前的请求总数</span></div><div class="line">    <span class="keyword">long</span> updatedErrorCount = errorCount; <span class="comment">// 之前的失败个数</span></div><div class="line"></div><div class="line">    <span class="keyword">long</span> successCount = eventTypeCounts[HystrixEventType.SUCCESS.ordinal()];</div><div class="line">    <span class="keyword">long</span> failureCount = eventTypeCounts[HystrixEventType.FAILURE.ordinal()];</div><div class="line">    <span class="keyword">long</span> timeoutCount = eventTypeCounts[HystrixEventType.TIMEOUT.ordinal()];</div><div class="line">    <span class="keyword">long</span> threadPoolRejectedCount = eventTypeCounts[HystrixEventType.THREAD_POOL_REJECTED.ordinal()];</div><div class="line">    <span class="keyword">long</span> semaphoreRejectedCount = eventTypeCounts[HystrixEventType.SEMAPHORE_REJECTED.ordinal()];</div><div class="line"></div><div class="line">    <span class="comment">// 加上所有事件的总数</span></div><div class="line">    updatedTotalCount += (successCount + failureCount + timeoutCount + threadPoolRejectedCount + semaphoreRejectedCount);</div><div class="line">    <span class="comment">// 加上失败事件的总数（包括请求失败、超时、线程池满、信号量满）</span></div><div class="line">    updatedErrorCount += (failureCount + timeoutCount + threadPoolRejectedCount + semaphoreRejectedCount);</div><div class="line">    <span class="keyword">return</span> <span class="keyword">new</span> HealthCounts(updatedTotalCount, updatedErrorCount);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>Hystrix 熔断器里会实时地去消费每个窗口产生的健康统计数据，并根据指标来决定熔断器的状态：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div></pre></td><td class="code"><pre><div class="line"><span class="comment">/* package */</span><span class="class"><span class="keyword">class</span> <span class="title">HystrixCircuitBreakerImpl</span> <span class="keyword">implements</span> <span class="title">HystrixCircuitBreaker</span> </span>&#123;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> HystrixCommandProperties properties;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> HystrixCommandMetrics metrics;</div><div class="line"></div><div class="line">    <span class="keyword">enum</span> Status &#123;</div><div class="line">        CLOSED, OPEN, HALF_OPEN;</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> AtomicReference&lt;Status&gt; status = <span class="keyword">new</span> AtomicReference&lt;Status&gt;(Status.CLOSED);</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> AtomicLong circuitOpened = <span class="keyword">new</span> AtomicLong(-<span class="number">1</span>);</div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> AtomicReference&lt;Subscription&gt; activeSubscription = <span class="keyword">new</span> AtomicReference&lt;Subscription&gt;(<span class="keyword">null</span>);</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">protected</span> <span class="title">HystrixCircuitBreakerImpl</span><span class="params">(HystrixCommandKey key, HystrixCommandGroupKey commandGroup, <span class="keyword">final</span> HystrixCommandProperties properties, HystrixCommandMetrics metrics)</span> </span>&#123;</div><div class="line">        <span class="keyword">this</span>.properties = properties;</div><div class="line">        <span class="keyword">this</span>.metrics = metrics;</div><div class="line"></div><div class="line">        <span class="comment">//On a timer, this will set the circuit between OPEN/CLOSED as command executions occur</span></div><div class="line">        Subscription s = subscribeToStream();</div><div class="line">        activeSubscription.set(s);</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">private</span> Subscription <span class="title">subscribeToStream</span><span class="params">()</span> </span>&#123;</div><div class="line">        <span class="comment">/*</span></div><div class="line">         * This stream will recalculate the OPEN/CLOSED status on every onNext from the health stream</div><div class="line">         */</div><div class="line">        <span class="keyword">return</span> metrics.getHealthCountsStream()</div><div class="line">                .observe()</div><div class="line">                .subscribe(<span class="keyword">new</span> Subscriber&lt;HealthCounts&gt;() &#123;</div><div class="line">                    <span class="meta">@Override</span></div><div class="line">                    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">onCompleted</span><span class="params">()</span> </span>&#123;</div><div class="line"></div><div class="line">                    &#125;</div><div class="line"></div><div class="line">                    <span class="meta">@Override</span></div><div class="line">                    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">onError</span><span class="params">(Throwable e)</span> </span>&#123;</div><div class="line"></div><div class="line">                    &#125;</div><div class="line"></div><div class="line">                    <span class="meta">@Override</span></div><div class="line">                    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">onNext</span><span class="params">(HealthCounts hc)</span> </span>&#123;</div><div class="line">                        <span class="comment">// check if we are past the statisticalWindowVolumeThreshold</span></div><div class="line">                        <span class="keyword">if</span> (hc.getTotalRequests() &lt; properties.circuitBreakerRequestVolumeThreshold().get()) &#123;</div><div class="line">                            <span class="comment">// we are not past the minimum volume threshold for the stat window,</span></div><div class="line">                            <span class="comment">// so no change to circuit status.</span></div><div class="line">                            <span class="comment">// if it was CLOSED, it stays CLOSED</span></div><div class="line">                            <span class="comment">// if it was half-open, we need to wait for a successful command execution</span></div><div class="line">                            <span class="comment">// if it was open, we need to wait for sleep window to elapse</span></div><div class="line">                        &#125; <span class="keyword">else</span> &#123;</div><div class="line">                            <span class="keyword">if</span> (hc.getErrorPercentage() &lt; properties.circuitBreakerErrorThresholdPercentage().get()) &#123;</div><div class="line">                                <span class="comment">//we are not past the minimum error threshold for the stat window,</span></div><div class="line">                                <span class="comment">// so no change to circuit status.</span></div><div class="line">                                <span class="comment">// if it was CLOSED, it stays CLOSED</span></div><div class="line">                                <span class="comment">// if it was half-open, we need to wait for a successful command execution</span></div><div class="line">                                <span class="comment">// if it was open, we need to wait for sleep window to elapse</span></div><div class="line">                            &#125; <span class="keyword">else</span> &#123;</div><div class="line">                                <span class="comment">// our failure rate is too high, we need to set the state to OPEN</span></div><div class="line">                                <span class="keyword">if</span> (status.compareAndSet(Status.CLOSED, Status.OPEN)) &#123;</div><div class="line">                                    circuitOpened.set(System.currentTimeMillis());</div><div class="line">                                &#125;</div><div class="line">                            &#125;</div><div class="line">                        &#125;</div><div class="line">                    &#125;</div><div class="line">                &#125;);</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<h1 id="总结">总结</h1><p>Hystrix 1.5 使用 RxJava 1.x 来实现滑动窗口，将滑动窗口抽象成响应式数据流的形式，既适合 Hystrix 事件驱动的特点，又易于实现和使用。滑动窗口的实现的要点就是每个桶的聚合以及滑动窗口的形成，Hystrix 巧妙地运用了 RxJava 中的 <a href="http://reactivex.io/documentation/operators/window.html" target="_blank" rel="external"><code>window</code></a> 操作符来将单位窗口时间内的事件，以及将一个窗口大小内的桶聚集到一起，并通过 <a href="http://reactivex.io/documentation/operators/reduce.html" target="_blank" rel="external"><code>reduce</code></a> 等折叠操作将事件集合聚集为桶，将滑动窗口内的桶聚集成指标数据，非常巧妙。同时，Hystrix 利用 ThreadLocal<subject> 作为一个线程安全的“代理”，可以确保多个发布者写的线程安全；通过 RxJava 的 <a href="http://reactivex.io/documentation/operators/refcount.html" target="_blank" rel="external"><code>share</code></a> 操作符可以确保多个订阅者从某个共享的 Observable 中观察的序列一致。</subject></p>
<p>最后用一张图来总结 Hystrix Metrics 事件驱动的流程：</p>
<p><img src="/assets/blog-img-k1/hystrix-metrics-event-driven-flow.png" alt="Event driven flow of Hystrix"></p>
<hr>
<h1 id="References">References</h1><ul>
<li><a href="https://github.com/Netflix/Hystrix/wiki/Metrics-and-Monitoring" target="_blank" rel="external">Metrics and Monitoring - Hystrix Wiki</a></li>
<li><a href="https://github.com/Netflix/Hystrix/wiki/How-it-Works" target="_blank" rel="external">How it Works - Hystrix Wiki</a></li>
<li><a href="http://reactivex.io/documentation/operators.html" target="_blank" rel="external">ReactiveX Operators</a></li>
</ul>
]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;总览&quot;&gt;总览&lt;/h1&gt;&lt;p&gt;Netflix Hystrix 通过类似滑动窗口的数据结构来统计调用的指标数据。Hystrix 1.5 将滑动窗口设计成了数据流（reactive stream, RxJava 中的 &lt;code&gt;Observable&lt;/code&gt;）的形
    
    </summary>
    
      <category term="高可用架构" scheme="http://www.sczyh30.com/categories/%E9%AB%98%E5%8F%AF%E7%94%A8%E6%9E%B6%E6%9E%84/"/>
    
    
      <category term="Netflix Hystrix" scheme="http://www.sczyh30.com/tags/Netflix-Hystrix/"/>
    
      <category term="高可用架构" scheme="http://www.sczyh30.com/tags/%E9%AB%98%E5%8F%AF%E7%94%A8%E6%9E%B6%E6%9E%84/"/>
    
      <category term="限流降级" scheme="http://www.sczyh30.com/tags/%E9%99%90%E6%B5%81%E9%99%8D%E7%BA%A7/"/>
    
      <category term="滑动窗口" scheme="http://www.sczyh30.com/tags/%E6%BB%91%E5%8A%A8%E7%AA%97%E5%8F%A3/"/>
    
      <category term="RxJava" scheme="http://www.sczyh30.com/tags/RxJava/"/>
    
  </entry>
  
  <entry>
    <title>call/cc 总结 | Scheme</title>
    <link href="http://www.sczyh30.com/posts/Functional-Programming/call-with-current-continuation/"/>
    <id>http://www.sczyh30.com/posts/Functional-Programming/call-with-current-continuation/</id>
    <published>2016-10-30T16:00:00.000Z</published>
    <updated>2018-09-23T12:19:04.000Z</updated>
    
    <content type="html"><![CDATA[<h1 id="Continuation">Continuation</h1><p>Continuation 也是一个老生常谈的东西了，我们来回顾一下。首先我们看一下 <em>TSPL4</em> 中定义的表达式求值需要做的事：</p>
<blockquote>
<p>During the evaluation of a Scheme expression, the implementation must keep track of two things: (1) what to evaluate and (2) what to do with the value.</p>
</blockquote>
<p>Continuation 即为其中的(2)，即表达式被求值以后，<strong>接下来要对表达式做的计算</strong>。<em>R5RS</em> 中 continuation 的定义为：</p>
<blockquote>
<p>The continuation represents an entire (default) future for the computation.</p>
</blockquote>
<p>比如 <code>(+ (* 2 3) (+ 1 7))</code> 表达式中，<code>(* 2 3)</code>的 continuation 为：保存 <code>(* 2 3)</code> 计算出的值 <code>6</code>，然后计算 <code>(+ 1 7)</code> 的值，最后将两表达式的值相加，结束；<code>(+ 1 7)</code> 的 continuation 为：保存 <code>(+ 1 7)</code> 的值 <code>8</code>，将其与前面计算出的 <code>6</code> 相加，结束。</p>
<p>Scheme 中的 continuation 是 first-class 的，也就是说它可以被当做参数进行传递和返回；并且 Scheme 中可以将 continuation 视为一个 procedure，也就是说可以调用 continuation 执行后续的运算。</p>
<h1 id="call/cc">call/cc</h1><p>每个表达式在求值的时候，都会有一个对应的 <strong>current continuation</strong>，它在等着当前表达式求值完毕然后把值传递给它。那么如何捕捉 current continuation 呢？这就要用到 Scheme 中强大的 <code>call/cc</code> 了。<code>call/cc</code> 的全名是 <code>call-with-current-continuation</code> ，它可以捕捉当前环境下的 current continuation 并利用它做各种各样的事情，如改变控制流，实现非本地退出(non-local exit)、协程(coroutine)、多任务(multi-tasking)等，非常方便。注意这里的 continuation 将当前 context 一并打包保存起来了，而不只是保存程序运行的位置。下面我们来举几个例子说明一下 <code>call/cc</code> 的用法。</p>
<h2 id="current_continuation">current continuation</h2><p>我们先来看个最简单的例子 —— 用它来捕捉 current continuation 并作为 procedure 调用。<code>call/cc</code> 接受一个函数，该函数接受一个参数，此参数即为 current continuation。以之前 <code>(+ (* 2 3) (+ 1 7))</code> 表达式中 <code>(* 2 3)</code> 的 continuation 为例：</p>
<figure class="highlight scheme"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">(<span class="name"><span class="builtin-name">define</span></span> cc <span class="literal">#f</span>)</div><div class="line">(<span class="name"><span class="builtin-name">+</span></span> (<span class="name"><span class="builtin-name">call/cc</span></span> (<span class="name"><span class="builtin-name">lambda</span></span> (return)</div><div class="line">                (<span class="name"><span class="builtin-name">set!</span></span> cc return)</div><div class="line">                (<span class="name"><span class="builtin-name">*</span></span> <span class="number">2</span> <span class="number">3</span>)))</div><div class="line">   (<span class="name"><span class="builtin-name">+</span></span> <span class="number">1</span> <span class="number">7</span>))</div></pre></td></tr></table></figure>
<p>我们将 <code>(* 2 3)</code> 的 current continuation (用<code>(+ ? (+ 1 7))</code>表示) 绑定给 <code>cc</code> 变量。现在 <code>cc</code> 就对应了一个 continuation ，它相当于过程 <code>(define (cc x) (+ (x) (+ 1 7)))</code>，等待一个值然后进行后续的运算：</p>
<figure class="highlight"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line">&gt; cc</div><div class="line">#&lt;continuation&gt;</div><div class="line">&gt; (cc 10)</div><div class="line">18</div><div class="line">&gt; (cc (* 2 3))</div><div class="line">14</div></pre></td></tr></table></figure>
<p>这个例子很好理解，我们下面引入 <code>call/cc</code> 的本质 —— 控制流变换。在 Scheme 中，假设 <code>call/cc</code> 捕捉到的 current continuation 为 <code>cc</code>(位于 <code>lambda</code> 中)，如果 <code>cc</code> 作为过程 <strong>直接或间接地被调用</strong>（即给它传值），<code>call/cc</code> 会立即返回，返回值即为传入 <code>cc</code> 的值。即一旦 current continuation 被调用，控制流会跳到 <code>call/cc</code> 处。因此，利用 <code>call/cc</code>，我们可以摆脱顺序执行的限制，在程序中跳来跳去，非常灵活。下面我们举几个 non-local exit 的例子来说明。</p>
<h2 id="Non-local_exit">Non-local exit</h2><p>Scheme 中没有 <code>break</code> 和 <code>return</code> 关键字，因此在循环中如果想 <code>break</code> 并提前返回的话就得借助 <code>call/cc</code>。比如下面的例子寻找传入的 <code>list</code> 中是否包含 <code>5</code>：</p>
<figure class="highlight scheme"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line">(<span class="name"><span class="builtin-name">define</span></span> (<span class="name">do-with</span> element return)</div><div class="line">  (<span class="name"><span class="builtin-name">if</span></span> (<span class="name"><span class="builtin-name">=</span></span> element <span class="number">5</span>)</div><div class="line">      (<span class="name">return</span> <span class="symbol">'find-five</span>)</div><div class="line">      (<span class="name">void</span>)))</div><div class="line"></div><div class="line">(<span class="name"><span class="builtin-name">define</span></span> (<span class="name">check-lst</span> lst)</div><div class="line">  (<span class="name"><span class="builtin-name">call/cc</span></span> (<span class="name"><span class="builtin-name">lambda</span></span> (return)</div><div class="line">             (<span class="name"><span class="builtin-name">for-each</span></span> (<span class="name"><span class="builtin-name">lambda</span></span> (element)</div><div class="line">                        (<span class="name">do-with</span> element return)</div><div class="line">                        (<span class="name">printf</span> <span class="string">"~a~%"</span> element))</div><div class="line">                      lst)</div><div class="line">             <span class="symbol">'not-found</span>)))</div></pre></td></tr></table></figure>
<p>测试：</p>
<figure class="highlight"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line">&gt; (check-lst '(0 2 4))</div><div class="line">0</div><div class="line">2</div><div class="line">4</div><div class="line">'not-found</div><div class="line">&gt; (check-lst '(0 3 5 1))</div><div class="line">0</div><div class="line">3</div><div class="line">'find-five</div></pre></td></tr></table></figure>
<p><code>check-lst</code> 过程会遍历列表中的元素，每次都会将 current continuation 传给 <code>do-with</code> 过程并进行调用，一旦<code>do-with</code> 遇到 <code>5</code>，我们就将结果传给 current continuation (即 <code>return</code>)，此时控制流会马上跳回 <code>check-lst</code> 过程中的 <code>call/cc</code> 处，这时候就已经终止遍历了（跳出了循环）。<code>call/cc</code> 的返回值为 <code>&#39;find-five</code>，所以最后会在控制台上打印出 <code>&#39;find-five</code>。</p>
<p>我们再来看一个经典的 generator 的例子，它非常像 Python 和 ES 6 中的 <code>yield</code>，每次调用的时候都会返回 list 中的一个元素：</p>
<figure class="highlight scheme"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div></pre></td><td class="code"><pre><div class="line">(<span class="name"><span class="builtin-name">define</span></span> (<span class="name">generate-one-element-at-a-time</span> lst)</div><div class="line"></div><div class="line">  <span class="comment">;; Hand the next item from a-list to "return" or an end-of-list marker</span></div><div class="line">  (<span class="name"><span class="builtin-name">define</span></span> (<span class="name">control-state</span> return)</div><div class="line">    (<span class="name"><span class="builtin-name">for-each</span></span></div><div class="line">     (<span class="name"><span class="builtin-name">lambda</span></span> (element)</div><div class="line">               (<span class="name"><span class="builtin-name">set!</span></span> return (<span class="name"><span class="builtin-name">call/cc</span></span></div><div class="line">                              (<span class="name"><span class="builtin-name">lambda</span></span> (resume-here)</div><div class="line">                                <span class="comment">;; Grab the current continuation</span></div><div class="line">                               (<span class="name"><span class="builtin-name">set!</span></span> control-state resume-here)</div><div class="line">                               (<span class="name">return</span> element)))))</div><div class="line">     lst)</div><div class="line">    (<span class="name">return</span> <span class="symbol">'you-fell-off-the-end</span>))</div><div class="line"></div><div class="line">  <span class="comment">;; This is the actual generator, producing one item from a-list at a time</span></div><div class="line">  (<span class="name"><span class="builtin-name">define</span></span> (<span class="name">generator</span>)</div><div class="line">    (<span class="name"><span class="builtin-name">call/cc</span></span> control-state))</div><div class="line"></div><div class="line">  <span class="comment">;; Return the generator</span></div><div class="line">  generator)</div><div class="line"></div><div class="line">(<span class="name"><span class="builtin-name">define</span></span> generate-digit</div><div class="line">  (<span class="name">generate-one-element-at-a-time</span> '(<span class="number">0</span> <span class="number">1</span> <span class="number">2</span>)))</div></pre></td></tr></table></figure>
<p>调用：</p>
<figure class="highlight"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">&gt; (generate-digit)</div><div class="line">0</div><div class="line">&gt; (generate-digit)</div><div class="line">1</div><div class="line">&gt; (generate-digit)</div><div class="line">2</div><div class="line">&gt; (generate-digit)</div><div class="line">'you-fell-off-the-end</div></pre></td></tr></table></figure>
<p>注意到这个例子里有两个 <code>call/cc</code>，大家刚看到的时候可能会有点晕，其实这两个 <code>call/cc</code> 各司其职，互不干扰。第一个 <code>call/cc</code> 负责保存遍历的状态（从此处恢复），而 <code>generator</code> 中的 <code>call/cc</code> 才是真正生成值的地方（非本地退出）。其中一个需要注意的地方就是 <code>control-state</code>，它在第一次调用的时候还是个 procedure，在第一次调用的过程中它就被重新绑定成一个 <code>continuation</code>，之后再调用 <code>generator</code> 生成器的时候，控制流就可以跳到之前遍历的位置继续执行下面的过程，从而达到生成器的效果。</p>
<h1 id="阴阳谜题">阴阳谜题</h1><p>continuation 环境嵌套。后面有时间专开一篇分析～</p>
<figure class="highlight scheme"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">(<span class="name"><span class="builtin-name">let*</span></span> ((<span class="name">yin</span></div><div class="line">         ((<span class="name"><span class="builtin-name">lambda</span></span> (cc) (<span class="name"><span class="builtin-name">display</span></span> <span class="literal">#\@</span>) cc) (<span class="name"><span class="builtin-name">call-with-current-continuation</span></span> (<span class="name"><span class="builtin-name">lambda</span></span> (c) c))))</div><div class="line">       (<span class="name">yang</span></div><div class="line">         ((<span class="name"><span class="builtin-name">lambda</span></span> (cc) (<span class="name"><span class="builtin-name">display</span></span> <span class="literal">#\*</span>) cc) (<span class="name"><span class="builtin-name">call-with-current-continuation</span></span> (<span class="name"><span class="builtin-name">lambda</span></span> (c) c)))))</div><div class="line">    (<span class="name">yin</span> yang))</div></pre></td></tr></table></figure>
<h1 id="call/cc与数理逻辑">call/cc与数理逻辑</h1><p>这里简单提一下 <code>call/cc</code> 与类型系统和数理逻辑的联系。<code>call/cc</code> 的类型是 <code>((P → Q) → P) → P</code>，通过 Curry-Howard 同构，它可以对应到经典逻辑中的 <strong>Peirce’s law</strong>：</p>
<p>$$((P \to Q) \to P) \to P$$</p>
<p>Peirce’s law 代表排中律 $P \land \lnot P$，这条逻辑无法在 λ演算所对应的直觉逻辑中表示（直觉逻辑中双重否定不成立），因此 <code>call/cc</code> 无法用 λ表达式定义。通常我们用扩展后的 $\lambda \mu \ calculus$ 来定义 <code>call/cc</code>，$\lambda \mu \ calculus$ 经 Curry-Howard 同构可以得到经典逻辑。</p>
<hr>
<h1 id="References">References</h1><ul>
<li><a href="http://www.scheme.com/tspl4/" target="_blank" rel="external">The Scheme Programming Language, 4th Edition</a></li>
<li><a href="http://community.schemewiki.org/?call-with-current-continuation" target="_blank" rel="external">Short introduction to call-with-current-continuation</a></li>
<li><a href="http://www.schemers.org/Documents/Standards/R5RS/HTML/" target="_blank" rel="external">The Revised Revised Revised Revised Revised Report on the Algorithmic Language Scheme (R5RS)</a></li>
<li><a href="https://docs.racket-lang.org/guide/conts.html" target="_blank" rel="external">The Racket Guide, 10.3 Continuations</a></li>
<li><a href="https://en.wikipedia.org/wiki/Call-with-current-continuation" target="_blank" rel="external">Call-with-current-continuation, Wikipedia</a></li>
<li><a href="https://golem.ph.utexas.edu/category/2008/01/the_continuation_passing_trans.html#c044523" target="_blank" rel="external">The Continuation Passing Transform and the Yoneda Embedding</a></li>
<li><a href="https://www.quora.com/What-is-continuation-passing-style-in-functional-programming" target="_blank" rel="external">What is continuation-passing style in functional programming? - Quora</a></li>
</ul>
]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;Continuation&quot;&gt;Continuation&lt;/h1&gt;&lt;p&gt;Continuation 也是一个老生常谈的东西了，我们来回顾一下。首先我们看一下 &lt;em&gt;TSPL4&lt;/em&gt; 中定义的表达式求值需要做的事：&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Durin
    
    </summary>
    
      <category term="Functional Programming" scheme="http://www.sczyh30.com/categories/Functional-Programming/"/>
    
    
      <category term="Scheme" scheme="http://www.sczyh30.com/tags/Scheme/"/>
    
      <category term="Continuation" scheme="http://www.sczyh30.com/tags/Continuation/"/>
    
      <category term="CPS" scheme="http://www.sczyh30.com/tags/CPS/"/>
    
      <category term="PLT" scheme="http://www.sczyh30.com/tags/PLT/"/>
    
      <category term="Mathematical Logic" scheme="http://www.sczyh30.com/tags/Mathematical-Logic/"/>
    
  </entry>
  
  <entry>
    <title>分布式计算系统中的 GC 问题 | Yak (OSDI 2016) 学习笔记</title>
    <link href="http://www.sczyh30.com/posts/Distributed-System/distributed-computing-system-gc-osdi16-yak/"/>
    <id>http://www.sczyh30.com/posts/Distributed-System/distributed-computing-system-gc-osdi16-yak/</id>
    <published>2016-10-16T16:00:00.000Z</published>
    <updated>2018-09-23T12:17:55.000Z</updated>
    
    <content type="html"><![CDATA[<p>最近一直在关注 OSDI 2016，发现了这篇关于分布式计算系统内存管理的论文：<a href="https://www.usenix.org/system/files/conference/osdi16/osdi16-nguyen.pdf" target="_blank" rel="external">Yak: A High-Performance Big-Data-Friendly. Garbage Collector</a>，感觉比较有趣，拿来总结总结~</p>
<p>Yak 是一个 JVM 平台上的针对大数据场景（分布式计算框架）设计、优化的 Garbage Collector。</p>
<h1 id="背景">背景</h1><p>在传统的基于分代模型的垃圾回收算法中，小对象首先被划分到 Young Generation（新生代）；如果对象经历过一定阈值的 GC 还会存活后，它就会被晋升至 Old Generation（老年代）；大对象可以直接进入老年代。</p>
<p>对于分布式计算框架而言，这样的模型有一定的弊端：没有考虑分布式计算情景下对象的生命周期。分布式计算中控制过程(Control Path)与数据处理过程(Data Path)界限明显，如果全都用统一的 GC 模型的话，会导致频繁请求 GC 数据，扫描全堆，最后实际回收的很少，导致 Full GC (STW)。因此，像 Apache Spark 在 1.5 版本以后已经放弃使用 JVM 的 GC 进行内存管理，而是直接利用 unsafe 包进行内存管理。这样十分麻烦还容易出错。</p>
<p>Yak 就是为了解决这样的问题而诞生的。既然分布式计算过程中控制过程与数据处理过程界限明显，Yak 针对这两种过程中的数据划分了两种不同的空间(space):</p>
<ul>
<li>Control Space (CS)</li>
<li>Data Space (DS)</li>
</ul>
<p>对于控制过程（比如任务的调度、日志记录等），其内存布局与传统的一致，GC 还是采用分代模型，分 YoungGen/OldGen/Metaspace。控制过程产生的对象小、生命周期短暂，符合分代假设。</p>
<p>而对于数据处理过程，其中的对象通常都是很大的、在计算周期中一直需要访问的，因此 Yak 提出了 Epoch Region，数据对象的生命周期依赖于每个 epoch。每个 epoch 的 start 与 end 需要用户来设置（但是很简单）。</p>
<h1 id="时域抽象">时域抽象</h1><blockquote>
<p>Epoch hypothesis: many data-path objects have the same life span and can be reclaimed together at the end of an epoch.</p>
</blockquote>
<p>时域抽象(Epoch Region): 抽象成 semilattice(半格)，用于描述 nested epoches 之间的偏序关系。见论文 Figure 5。(Order Theory在这里非常有用)</p>
<p>如何正确地回收某个特定的 Region</p>
<p>如果有的对象生命周期超出此 epoch，如何将其迁移至“安全地带”？</p>
<ul>
<li>思考点1：标记 escaping objects</li>
<li>思考点2：决定 escaping objects 的迁移终点并且执行复制</li>
</ul>
<p>对于标记的过程，可以以 cross-region/space references 为根节点来遍历对象图并且标记其中的 escaping objects（传递闭包）</p>
<p>对于决定其 destination 的过程，需要计算出对象 O 的引用 region 的上确界(via semilattice)。</p>
<p>如果对应的 region 具有继承关系，则应选择最上面的（上确界）。如果是不同线程执行的，那么对应的上确界则为 CS。</p>
<blockquote>
<p>TODO: 待详细总结</p>
</blockquote>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;最近一直在关注 OSDI 2016，发现了这篇关于分布式计算系统内存管理的论文：&lt;a href=&quot;https://www.usenix.org/system/files/conference/osdi16/osdi16-nguyen.pdf&quot; target=&quot;_blank&quot;
    
    </summary>
    
      <category term="Distributed System" scheme="http://www.sczyh30.com/categories/Distributed-System/"/>
    
    
      <category term="分布式系统" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="分布式计算" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E8%AE%A1%E7%AE%97/"/>
    
      <category term="GC" scheme="http://www.sczyh30.com/tags/GC/"/>
    
      <category term="Paper" scheme="http://www.sczyh30.com/tags/Paper/"/>
    
  </entry>
  
  <entry>
    <title>Distributed System | Spark RDD 论文总结</title>
    <link href="http://www.sczyh30.com/posts/Distributed-System/spark-rdd-paper-summary/"/>
    <id>http://www.sczyh30.com/posts/Distributed-System/spark-rdd-paper-summary/</id>
    <published>2016-09-26T16:00:00.000Z</published>
    <updated>2018-09-23T14:15:44.000Z</updated>
    
    <content type="html"><![CDATA[<p>本篇文章是对Spark RDD论文的总结，中间会穿插一些Spark的内部实现总结，对应Spark版本为2.0。</p>
<h1 id="RDD">RDD</h1><h2 id="Motivation">Motivation</h2><p>传统的分布式计算框架(如MapReduce)在执行计算任务时，中间结果通常会存于磁盘中，这样带来的IO消耗是非常大的，尤其是对于各种机器学习算法，它们需要复用上次计算的结果进行迭代，如果每次结果都存到磁盘上再从磁盘读取，耗时会很大。因此Spark这篇论文提出了一种新的分布式数据抽象 —— <strong>RDD</strong>。</p>
<h2 id="设计思想及特点">设计思想及特点</h2><p><strong>Resilient Distributed Dataset</strong>(RDD)是Apache Spark中数据的核心抽象，它是一种只读的、分区的数据记录集合。</p>
<p>RDD的特点：</p>
<ul>
<li>Lazy evaluation，只在需要的时候才进行计算</li>
<li>RDD里面的数据是分区的，每一块数据都可能分布在集群内不同的节点上；支持并行计算</li>
<li>Resilient: 借助RDD lineage graph，Spark可以重新执行之前失败的计算任务而不用整体上重新计算，保证了容错性而且非常灵活，实现了fault-tolerance</li>
</ul>
<p>那么如何操作、处理数据呢？Spark提供了一组函数式编程风格的API，可以很方便地对RDD进行操作、变换，就像操作集合一样。比如：</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">val</span> rdd = sc.parallelize(<span class="number">1</span> to <span class="number">100</span>)</div><div class="line"></div><div class="line"><span class="keyword">val</span> result = rdd.map(_ + <span class="number">10</span>)</div><div class="line">  .filter(_ &gt; <span class="number">15</span>)</div><div class="line">  .map(x =&gt; (x, <span class="number">1</span>))</div><div class="line">  .reduceByKey(_+_)</div><div class="line">  .collect</div></pre></td></tr></table></figure>
<p>并且开发者可以根据需要自己编写相应的RDD以及RDD之间的操作，非常方便。可以这么理解，RDD就相当于抽象的数据表示，而operation就相当于一套DSL用于对RDD进行变换或者求值。</p>
<h2 id="RDD的表示">RDD的表示</h2><p>Spark中的RDD主要包含五部分信息：</p>
<ul>
<li><code>partitions()</code>: partition集合</li>
<li><code>dependencies()</code>: 当前RDD的dependency集合</li>
<li><code>iterator(split, context)</code>: 对每个partition进行计算或读取操作的函数</li>
<li><code>partitioner()</code>: 分区方式，如<code>HashPartitioner</code>和<code>RangePartitioner</code></li>
<li><code>preferredLocations(split)</code>: 访问某个partition最快的节点</li>
</ul>
<p>所有的RDD都继承抽象类<code>RDD</code>。几种常见的操作：</p>
<ul>
<li><code>sc#textFile</code>: 生成<code>HadoopRDD</code>，代表可以从HDFS中读取数据的RDD</li>
<li><code>sc#parallelize</code>: 生成<code>ParallelCollectionRDD</code>，代表从Scala集合中生成的RDD</li>
</ul>
<ul>
<li><code>map</code>, <code>flatMap</code>, <code>filter</code>: 生成<code>MapPartitionsRDD</code>，其partition与parent RDD一致，同时会对parent RDD中<code>iterator</code>函数返回的数据进行对应的操作(lazy)</li>
<li><code>union</code>: 生成<code>UnionRDD</code>或<code>PartitionerAwareUnionRDD</code></li>
<li><code>reduceByKey</code>, <code>groupByKey</code>: 生成<code>ShuffledRDD</code>，需要进行shuffle操作</li>
<li><code>cogroup</code>, <code>join</code>: 生成<code>CoGroupedRDD</code></li>
</ul>
<h1 id="Operations">Operations</h1><p>Spark里面对RDD的操作分为两种：<strong>transformation</strong> 和 <strong>action</strong>。</p>
<ul>
<li>transformation是lazy的，仅仅会保存计算步骤并返回一个新的RDD，而不会立刻执行计算操作</li>
<li>action会依次执行计算操作并且得到结果</li>
</ul>
<p>这些transformation和action在FP中应该是很常见的，如<code>map</code>, <code>flatMap</code>, <code>filter</code>, <code>reduce</code>, <code>count</code>, <code>sum</code>。</p>
<p>对单个数据操作的transformation函数都在<code>RDD</code>抽象类内，而对tuple操作的transformation都在<code>PairRDDFunctions</code>包装类中。<code>RDD</code>可以通过implicit函数在符合类型要求的时候自动转换为<code>PairRDDFunctions</code>类，从而可以进行<code>reduceByKey</code>之类的操作。对应的implicit函数：</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">implicit</span> <span class="function"><span class="keyword">def</span> <span class="title">rddToPairRDDFunctions</span></span>[<span class="type">K</span>, <span class="type">V</span>](rdd: <span class="type">RDD</span>[(<span class="type">K</span>, <span class="type">V</span>)])</div><div class="line">  (<span class="keyword">implicit</span> kt: <span class="type">ClassTag</span>[<span class="type">K</span>], vt: <span class="type">ClassTag</span>[<span class="type">V</span>], ord: <span class="type">Ordering</span>[<span class="type">K</span>] = <span class="literal">null</span>): <span class="type">PairRDDFunctions</span>[<span class="type">K</span>, <span class="type">V</span>] = &#123;</div><div class="line">  <span class="keyword">new</span> <span class="type">PairRDDFunctions</span>(rdd)</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<h1 id="Dependency">Dependency</h1><p>上面我们提到，RDD只会在需要的时候计算结果，调用那些transformation方法以后，对应的transformation信息只是被简单地存储起来，直到调用某个action才会真正地去执行计算。Spark中RDD之间是有联系的，RDD之间会形成依赖关系，也就是形成lineage graph(依赖图)。Dependency大致分两种：narrow dependency和wide dependency。</p>
<ul>
<li>Narrow dependency(<code>NarrowDependency</code>): Parent RDD中的每个partition最多被child RDD中的一个partition使用，即一对一的关系。比如<code>map</code>, <code>flatMap</code>, <code>filter</code>等transformation都是narrow dependency</li>
<li>Wide dependency(<code>ShuffleDependency</code>)：Parent RDD中的每个partition会被child RDD中的多个partition使用，即一对多的关系。比如<code>join</code>生成的RDD一般是wide dependency(不同的partitioner)</li>
</ul>
<p>论文中的图例很直观地表示了RDD间的依赖关系：</p>
<p><img src="/assets/blog-img-k1/spark-paper-dependency-image.png" alt="Spark RDD Dependency"></p>
<p>这样划分dependency的原因：</p>
<ol>
<li>Narrow dependency可以方便地以流水线的形式执行计算，即从头到尾一串chain下来。而wide dependency必须要等所有的parent RDD的结果都准备好以后再执行计算</li>
<li>Narrow dependency失败以后，Spark只需要重新计算失败的parent RDD即可；而对于wide dependency来说，一失败可能导致某些分区丢失，必须整体重新进行计算</li>
</ol>
<h1 id="Shuffle">Shuffle</h1><p>Spark中的shuffle操作与MapReduce中类似，在计算wide dependency对应的RDD的时候（即ShuffleMapStage）会触发。</p>
<p>首先来回顾一下为什么要进行shuffle操作。以<code>reduceByKey</code>操作为例，Spark要按照key把这些具有相同key的tuple聚集到一块然后进行计算操作。然而这些tuple可能在不同的partition中，甚至在不同的集群节点中，要想计算必须先把它们聚集起来。因此，Spark用一组map task来将每个分区写入到临时文件中，然后下一个stage端(reduce task)会根据编号获取临时文件，然后将partition中的tuple按照key聚集起来并且进行相应的操作。这里面还包括着排序操作（可能在map side也可能在reduce side进行）。</p>
<p>Shuffle是Spark的主要性能瓶颈之一（涉及磁盘IO,数据序列化和网络IO），其优化一直是个难题。</p>
<ul>
<li>Shuffle write(map task): <code>SortShuffleWriter#write</code></li>
<li>Shuffle read(reduce task): <code>ShuffleRDD#compute</code></li>
</ul>
<h1 id="Persistence">Persistence</h1><h2 id="Checkpointing">Checkpointing</h2><p>Checkpoint的目的是保存那些计算耗时较长的RDD数据(long lineage chains)，执行Checkpoint的时候会新提交一个Job，因此最好先<code>persist</code>后<code>checkpoint</code>。</p>
<h2 id="Cache/Persist">Cache/Persist</h2><p><code>cache</code>和<code>persist</code>用于缓存一些经常使用的RDD结果（但是不能太大）。</p>
<ul>
<li><code>persist</code>方法的主要作用是改变<code>StorageLevel</code>以在<code>compute</code>的时候通过<code>BlockManager</code>进行相应的持久化操作</li>
<li><code>cache</code>方法相当于设置存储级别为<code>MEMORY_ONLY</code></li>
</ul>
<h1 id="Job_Scheduling">Job Scheduling</h1><p>简单来说，Spark会将提交的计算划分为不同的stages，形成一个有向无环图(<strong>DAG</strong>)。Spark的调度器会按照DAG的次序依次进行计算每个stage，最终得到计算结果。执行计算的几个重要的类或接口如下：</p>
<ul>
<li><code>DAGScheduler</code></li>
<li><code>ActiveJob</code></li>
<li><code>Stage</code></li>
<li><code>Task</code></li>
<li><code>TaskScheduler</code></li>
<li><code>SchedulerBackend</code></li>
</ul>
<p>这里面最为重要的就是 <strong>DAGScheduler</strong> 了，它会将<strong>逻辑执行计划</strong>（即RDD lineage）转化为<strong>物理执行计划</strong>（stage/task）。之前我们提到过，当开发者对某个RDD执行action的时候，Spark才会执行真正的计算过程。当开发者执行action的时候，<code>SparkContext</code>会将当前的逻辑执行计划传给<code>DAGScheduler</code>，<code>DAGScheduler</code>会根据给定的逻辑执行计划生成一个Job(对应<code>ActiveJob</code>类)并提交。每执行一个acton都会生成一个<code>ActiveJob</code>。</p>
<p>提交Job的过程中，<code>DAGScheduler</code>会进行stage的划分。Spark里是按照<code>shuffle</code>操作来划分stage的，也就是说stage之间都是wide dependency，每个stage之内的dependency都是narrow dependency。这样划分的好处是尽可能地把多个narrow dependency的RDD放到同一个stage之内以便于进行pipeline计算，而wide dependency中child RDD必须等待所有的parent RDD计算完成并且<code>shuffle</code>以后才能接着计算，因此这样划分stage是最合适的。</p>
<p>划分好的stages会形成一个DAG，<code>DAGScheduler</code>会根据DAG中的顺序先提交parent stages（如果存在的话），再提交当前stage，以此类推，最先提交的是没有parent stage的stage。从执行角度来讲，一个stage的parent stages执行完以后，该stage才可以被执行。最后一个stage是产生最终结果的stage，对应<code>ResultStage</code>，而其余的stage都是<code>ShuffleMapStage</code>。下面是论文中stage划分的一个图例，非常直观：</p>
<p><img src="/assets/blog-img-k1/spark-paper-stage-figure.png" alt="DAG of stages"></p>
<p>提交stage的时候，Spark会根据stage的类型生成一组对应类型的<code>Task</code>(<code>ResultTask</code>或<code>ShuffleMapTask</code>)，然后将这些<code>Task</code>包装成<code>TaskSet</code>提交到<code>TaskScheduler</code>中。一个<code>Task</code>对应某个RDD中的某一个partition，即一个<code>Task</code>只负责某个partition的计算：</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">val</span> tasks: <span class="type">Seq</span>[<span class="type">Task</span>[_]] = <span class="keyword">try</span> &#123;</div><div class="line">  stage <span class="keyword">match</span> &#123;</div><div class="line">    <span class="keyword">case</span> stage: <span class="type">ShuffleMapStage</span> =&gt;</div><div class="line">      partitionsToCompute.map &#123; id =&gt;</div><div class="line">        <span class="keyword">val</span> locs = taskIdToLocations(id)</div><div class="line">        <span class="keyword">val</span> part = stage.rdd.partitions(id)</div><div class="line">        <span class="keyword">new</span> <span class="type">ShuffleMapTask</span>(stage.id, stage.latestInfo.attemptId,</div><div class="line">          taskBinary, part, locs, stage.latestInfo.taskMetrics, properties)</div><div class="line">      &#125;</div><div class="line"></div><div class="line">    <span class="keyword">case</span> stage: <span class="type">ResultStage</span> =&gt;</div><div class="line">      <span class="keyword">val</span> job = stage.activeJob.get</div><div class="line">      partitionsToCompute.map &#123; id =&gt;</div><div class="line">        <span class="keyword">val</span> p: <span class="type">Int</span> = stage.partitions(id)</div><div class="line">        <span class="keyword">val</span> part = stage.rdd.partitions(p)</div><div class="line">        <span class="keyword">val</span> locs = taskIdToLocations(id)</div><div class="line">        <span class="keyword">new</span> <span class="type">ResultTask</span>(stage.id, stage.latestInfo.attemptId,</div><div class="line">          taskBinary, part, locs, id, properties, stage.latestInfo.taskMetrics)</div><div class="line">      &#125;</div><div class="line">  &#125;</div><div class="line">&#125; <span class="keyword">catch</span> &#123;</div><div class="line">  <span class="comment">// 此处代码略...</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>TaskScheduler</code>会向执行任务的后端(<code>SchedulerBackend</code>，可以是Local, Mesos, Hadoop YARN或者其它集群管理组件)发送<code>ReviveOffers</code>消息，对应的执行后端接收到消息以后会将<code>Task</code>封装成<code>TaskRunner</code>(<code>Runnable</code>接口的实例)，然后提交到底层的<code>Executor</code>中，并行执行计算任务。</p>
<p><code>Executor</code>中的线程池定义如下：</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">val</span> threadPool = <span class="type">ThreadUtils</span>.newDaemonCachedThreadPool(<span class="string">"Executor task launch worker"</span>)</div></pre></td></tr></table></figure>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">newDaemonCachedThreadPool</span></span>(prefix: <span class="type">String</span>): <span class="type">ThreadPoolExecutor</span> = &#123;</div><div class="line">  <span class="keyword">val</span> threadFactory = namedThreadFactory(prefix)</div><div class="line">  <span class="type">Executors</span>.newCachedThreadPool(threadFactory).asInstanceOf[<span class="type">ThreadPoolExecutor</span>]</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到底层执行task的线程池实际上是JUC中的<code>CachedThreadPool</code>，按需创建新线程，同时会复用线程池中已经建好的线程。</p>
<p>最后用一幅图总结一下Job, Stage和Task的关系（图来自 <em>Mastering Apache Spark 2.0</em>）：</p>
<p><img src="/assets/blog-img-k1/spark-stage-tasks-jobs.png" alt="Stage, Job and Task in Spark"></p>
<p>整个Spark Context执行task的步骤图：</p>
<p><img src="/assets/blog-img-k1/spark-run-job-diagram.png" alt=""></p>
<h1 id="Memory_Management">Memory Management</h1><p>Spark中RDD的存储方式有两种：in memory和on disk，默认是in memory的。进行分布式计算的时候通常会读入大量的数据，并且通常还需要重用这些数据，如果简单地把内存管理交给GC的话，很容易导致回收失败从而cause full GC，影响性能。</p>
<p>Spark 1.5开始不再通过GC管理内存。Spark 1.5实现了一个内存管理器用于手动管理内存(Project Tungsten)，底层通过<code>Unsafe</code>类来直接分配和回收内存。</p>
<p>另外，分布式计算系统的GC方面还可以参考OSDI 2016的一篇论文: <em>Yak: A High-Performance Big-Data-Friendly Garbage Collector</em>。</p>
<h1 id="PageRank实例">PageRank实例</h1><p>下面在Spark中跑一个PageRank来观察一下生成的Stage DAG。PageRank的公式比较简单：</p>
<p>$$PageRank (p_i) = \frac{1-d}{N} + d \sum_{p_j \in M(p_i)} \frac{PageRank (p_j)}{L(p_j)} $$</p>
<p>这里我们选择damping factor=0.85，初始的rank值为1.0；PageRank算法可以用马尔科夫矩阵进行优化，但是这里迭代次数较小，可以直接进行迭代计算。对应代码：</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">val</span> iters = <span class="number">10</span></div><div class="line"><span class="keyword">val</span> data = sc.textFile(<span class="string">"data.txt"</span>)</div><div class="line"></div><div class="line"><span class="keyword">val</span> links = data.map &#123; s =&gt;</div><div class="line">  <span class="keyword">val</span> parts = s.split(<span class="string">"\\s+"</span>)</div><div class="line">  (parts(<span class="number">0</span>), parts(<span class="number">1</span>))</div><div class="line">&#125;.distinct().groupByKey().cache()</div><div class="line"><span class="keyword">var</span> ranks = links.mapValues(v =&gt; <span class="number">1.0</span>)</div><div class="line"></div><div class="line"><span class="keyword">for</span> (i &lt;- <span class="number">1</span> to iters) &#123;</div><div class="line">  <span class="keyword">val</span> contribs = links.join(ranks).values.flatMap &#123; <span class="keyword">case</span> (urls, rank) =&gt;</div><div class="line">    <span class="keyword">val</span> size = urls.size</div><div class="line">    urls.map(url =&gt; (url, rank / size))</div><div class="line">  &#125;</div><div class="line">  ranks = contribs.reduceByKey(_ + _).mapValues(<span class="number">0.15</span> + <span class="number">0.85</span> * _)</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">val</span> output = ranks.collect()</div></pre></td></tr></table></figure>
<p>对应的Stage DAG:</p>
<p><img src="/assets/blog-img-k1/spark-simple-rankpage-dag.png" alt="DAG of stages in PageRank Algorithm"></p>
<p>其中Stage 3中的RDD dependencies如下：</p>
<p><img src="/assets/blog-img-k1/spark-simple-rankpage-one-stage-dag.png" alt="One stage in PageRank Algorithm"></p>
<hr>
<h1 id="References">References</h1><ul>
<li><a href="http://people.csail.mit.edu/matei/papers/2012/nsdi_spark.pdf" target="_blank" rel="external">M. Zaharia, <em>et al</em>. <em>Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing</em>.</a></li>
<li><a href="https://research.google.com/archive/mapreduce-osdi04.pdf" target="_blank" rel="external">Jeffrey Dean, Sanjay Ghemawat. <em>MapReduce: Simplified Data Processing on Large Clusters</em>.</a></li>
<li><a href="https://www.gitbook.com/book/jaceklaskowski/mastering-apache-spark" target="_blank" rel="external">Mastering Apache Spark 2.0</a></li>
<li><a href="https://github.com/JerryLead/SparkInternals" target="_blank" rel="external">Spark Internals</a></li>
<li><a href="http://spark.apache.org/docs/latest/programming-guide.html" target="_blank" rel="external">Spark Programming Guide</a></li>
</ul>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;本篇文章是对Spark RDD论文的总结，中间会穿插一些Spark的内部实现总结，对应Spark版本为2.0。&lt;/p&gt;
&lt;h1 id=&quot;RDD&quot;&gt;RDD&lt;/h1&gt;&lt;h2 id=&quot;Motivation&quot;&gt;Motivation&lt;/h2&gt;&lt;p&gt;传统的分布式计算框架(如MapRed
    
    </summary>
    
      <category term="Distributed System" scheme="http://www.sczyh30.com/categories/Distributed-System/"/>
    
    
      <category term="分布式系统" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="分布式计算" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E8%AE%A1%E7%AE%97/"/>
    
      <category term="Paper" scheme="http://www.sczyh30.com/tags/Paper/"/>
    
      <category term="Spark" scheme="http://www.sczyh30.com/tags/Spark/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x 技术内幕 | 解道 Vert.x 线程模型</title>
    <link href="http://www.sczyh30.com/posts/Vert-x/vertx-advanced-demystifying-thread-model/"/>
    <id>http://www.sczyh30.com/posts/Vert-x/vertx-advanced-demystifying-thread-model/</id>
    <published>2016-09-24T16:00:00.000Z</published>
    <updated>2018-09-23T14:15:44.000Z</updated>
    
    <content type="html"><![CDATA[<h1 id="线程模型概述">线程模型概述</h1><p>Vert.x 的线程模型设计的非常巧妙。总的来说，Vert.x 中主要有两种线程：<strong>Event Loop 线程</strong> 和 <strong>Worker 线程</strong>。其中，Event Loop 线程结合了 Netty 的 <code>EventLoop</code>，用于处理事件。每一个 <code>EventLoop</code> 都与唯一的线程相绑定，这个线程就叫 Event Loop 线程。Event Loop 线程不能被阻塞，否则事件将无法被处理。</p>
<p>Worker 线程用于执行阻塞任务，这样既可以执行阻塞任务而又不阻塞 Event Loop 线程。</p>
<p>如果像 Node.js 一样只有单个 Event Loop 的话就不能充分利用多核 CPU 的性能了。为了充分利用多核 CPU 的性能，Vert.x 中提供了一组 Event Loop 线程。每个 Event Loop 线程都可以处理事件。为了保证线程安全，防止资源争用，Vert.x 保证了某一个 <code>Handler</code> 总是被同一个 Event Loop 线程执行，这样不仅可以保证线程安全，而且还可以在底层对锁进行优化提升性能。所以，只要开发者遵循 Vert.x 的线程模型，开发者就不需要再担心线程安全的问题，这是非常方便的。</p>
<p>本篇文章将底层的角度来解析 Vert.x 的线程模型。对应的 Vert.x 版本为 <strong>3.3.3</strong>。</p>
<h1 id="Event_Loop_线程">Event Loop 线程</h1><p>首先回顾一下 Event Loop 线程，它会不断地轮询获取事件，并将获取到的事件分发到对应的事件处理器中进行处理：</p>
<p><img src="/assets/blog-img-k1/vertx-event-loop-xx.png" alt="Vert.x Event Loop"></p>
<p>Vert.x 线程模型中最重要的一点就是：<strong>永远不要阻塞 Event Loop 线程</strong>。因为一旦处理事件的线程被阻塞了，事件就会一直积压着不能被处理，整个应用也就不能正常工作了。</p>
<p>Vert.x 中内置一种用于检测 Event Loop 是否阻塞的线程：<code>vertx-blocked-thread-checker</code>。一旦 Event Loop 处理某个事件的时间超过一定阈值（默认为 2000 ms）就会警告，如果阻塞的时间过长就会抛出异常。Block Checker 的实现原理比较简单，底层借助了 JUC 的 <code>TimerTask</code>，定时计算每个 Event Loop 线程的处理事件消耗的时间，如果超时就进行相应的警告。</p>
<h1 id="Vert-x_Thread">Vert.x Thread</h1><p>Vert.x 中的 Event Loop 线程及 Worker 线程都用 <code>VertxThread</code> 类表示，并通过 <code>VertxThreadFactory</code> 线程工厂来创建。<code>VertxThreadFactory</code> 创建 Vert.x 线程的过程非常简单：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> Thread <span class="title">newThread</span><span class="params">(Runnable runnable)</span> </span>&#123;</div><div class="line">  VertxThread t = <span class="keyword">new</span> VertxThread(runnable, prefix + threadCount.getAndIncrement(), worker, maxExecTime);</div><div class="line"></div><div class="line">  <span class="keyword">if</span> (checker != <span class="keyword">null</span>) &#123;</div><div class="line">    checker.registerThread(t);</div><div class="line">  &#125;</div><div class="line">  addToMap(t);</div><div class="line"></div><div class="line">  t.setDaemon(<span class="keyword">false</span>);</div><div class="line">  <span class="keyword">return</span> t;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>除了创建 <code>VertxThread</code> 线程之外，<code>VertxThreadFactory</code> 还会将此线程注册至 Block Checker 线程中以监视线程的阻塞情况，并且将此线程添加至内部的 <code>weakMap</code> 中。这个 <code>weakMap</code> 作用只有一个，就是在注销对应的 Verticle 的时候可以将每个 <code>VertxThread</code> 中的 <code>Context</code> 实例清除(unset)。为了保证资源不被一直占用，这里使用了 <code>WeakHashMap</code> 来存储每一个 <code>VertxThread</code>。当里面的 <code>VertxThread</code> 的引用不被其他实例持有的时候，它就会被标记为可清除的对象，等待 GC。</p>
<p>至于 <code>VertxThread</code>，它其实就是在普通线程的基础上存储了额外的数据（如对应的 Vert.x Context，最大执行时长，当前执行时间，是否为 Worker 线程等），这里就不多讲了。</p>
<h1 id="Vert-x_Context">Vert.x Context</h1><p>Vert.x 底层中一个重要的概念就是 <code>Context</code>，每个 <code>Context</code> 都会绑定着一个 Event Loop 线程（而一个 Event Loop 线程可以对应多个 <code>Context</code>）。我们可以把 <code>Context</code> 看作是控制一系列的 <code>Handler</code> 的执行作用域及顺序的上下文对象。</p>
<p>每当 Vert.x 底层将事件分发至 <code>Handler</code> 的时候，Vert.x 都会给此 <code>Handler</code> 绑定一个 <code>Context</code> 用于处理任务：</p>
<ul>
<li>如果当前线程是 Vert.x 线程(<code>VertxThread</code>)，那么 Vert.x 就会复用此线程上绑定的 <code>Context</code>；如果没有对应的 <code>Context</code> 就创建新的</li>
<li>如果当前线程是普通线程，就创建新的 <code>Context</code></li>
</ul>
<p>Vert.x 中存在三种 <code>Context</code>，与之前的线程种类相对应：</p>
<ul>
<li><code>EventLoopContext</code></li>
<li><code>WorkerContext</code></li>
<li><code>MultiThreadedWorkerContext</code></li>
</ul>
<h2 id="Event_Loop_Context">Event Loop Context</h2><p>每个 Event Loop Context 都会对应着唯一的一个 <code>EventLoop</code>，即一个 Event Loop Context 只会在同一个 Event Loop 线程上执行任务。在创建 <code>Context</code> 的时候，Vert.x 会自动根据轮询策略选择对应的 <code>EventLoop</code>:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">protected</span> <span class="title">ContextImpl</span><span class="params">(VertxInternal vertx, WorkerPool internalBlockingPool, WorkerPool workerPool, String deploymentID, JsonObject config,</span></span></div><div class="line">                        ClassLoader tccl) &#123;</div><div class="line">    <span class="comment">// ...</span></div><div class="line">    EventLoopGroup group = vertx.getEventLoopGroup();</div><div class="line">    <span class="keyword">if</span> (group != <span class="keyword">null</span>) &#123;</div><div class="line">      <span class="keyword">this</span>.eventLoop = group.next();</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">this</span>.eventLoop = <span class="keyword">null</span>;</div><div class="line">    &#125;</div><div class="line">    <span class="comment">// ...</span></div><div class="line">  &#125;</div></pre></td></tr></table></figure>
<p>在 Netty 中，<code>EventLoopGroup</code> 代表一组 <code>EventLoop</code>，而从中获取 <code>EventLoop</code> 的方法则是 <code>next</code> 方法。<code>EventLoopGroup</code> 中 <code>EventLoop</code> 的数量由 CPU 核数所确定。Vert.x 这里使用了 Netty NIO 对应的 <code>NioEventLoop</code>：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">eventLoopGroup = <span class="keyword">new</span> NioEventLoopGroup(options.getEventLoopPoolSize(), eventLoopThreadFactory);</div><div class="line">eventLoopGroup.setIoRatio(NETTY_IO_RATIO);</div></pre></td></tr></table></figure>
<p>对应的轮询算法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> EventExecutorChooser <span class="title">newChooser</span><span class="params">(EventExecutor[] executors)</span> </span>&#123;</div><div class="line">    <span class="keyword">if</span> (isPowerOfTwo(executors.length)) &#123;</div><div class="line">        <span class="keyword">return</span> <span class="keyword">new</span> PowerOfTowEventExecutorChooser(executors);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="keyword">return</span> <span class="keyword">new</span> GenericEventExecutorChooser(executors);</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到，正常情况下 Netty 会用轮询策略选择 <code>EventLoop</code>。特别地，如果 <code>EventLoop</code> 的个数是 2 的倍数的话，选择的会快一些：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="class"><span class="keyword">class</span> <span class="title">GenericEventExecutorChooser</span> <span class="keyword">implements</span> <span class="title">EventExecutorChooser</span> </span>&#123;</div><div class="line">    <span class="comment">// ...</span></div><div class="line">    <span class="meta">@Override</span></div><div class="line">    <span class="function"><span class="keyword">public</span> EventExecutor <span class="title">next</span><span class="params">()</span> </span>&#123;</div><div class="line">        <span class="keyword">return</span> executors[Math.abs(idx.getAndIncrement() % executors.length)];</div><div class="line">    &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="class"><span class="keyword">class</span> <span class="title">PowerOfTowEventExecutorChooser</span> <span class="keyword">implements</span> <span class="title">EventExecutorChooser</span> </span>&#123;</div><div class="line">    <span class="comment">// ...</span></div><div class="line">    <span class="meta">@Override</span></div><div class="line">    <span class="function"><span class="keyword">public</span> EventExecutor <span class="title">next</span><span class="params">()</span> </span>&#123;</div><div class="line">        <span class="keyword">return</span> executors[idx.getAndIncrement() &amp; executors.length - <span class="number">1</span>];</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们可以在 Embedded 模式下测试一下 Event Loop 线程的分配：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line">System.out.println(Thread.currentThread());</div><div class="line">Vertx vertx = Vertx.vertx();</div><div class="line"><span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; <span class="number">20</span>; i++) &#123;</div><div class="line">  <span class="keyword">int</span> index = i;</div><div class="line">  vertx.setTimer(<span class="number">1</span>, t -&gt; &#123;</div><div class="line">    System.out.println(index + <span class="string">":"</span> + Thread.currentThread());</div><div class="line">  &#125;);</div></pre></td></tr></table></figure>
<p>运行结果（不同机器运行顺序、Event Loop线程数可能不同）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div></pre></td><td class="code"><pre><div class="line">Thread[main,<span class="number">5</span>,main]</div><div class="line"><span class="number">0</span>:Thread[vert.x-eventloop-thread-<span class="number">0</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">1</span>:Thread[vert.x-eventloop-thread-<span class="number">1</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">2</span>:Thread[vert.x-eventloop-thread-<span class="number">2</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">3</span>:Thread[vert.x-eventloop-thread-<span class="number">3</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">5</span>:Thread[vert.x-eventloop-thread-<span class="number">5</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">6</span>:Thread[vert.x-eventloop-thread-<span class="number">6</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">8</span>:Thread[vert.x-eventloop-thread-<span class="number">8</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">7</span>:Thread[vert.x-eventloop-thread-<span class="number">7</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">10</span>:Thread[vert.x-eventloop-thread-<span class="number">10</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">9</span>:Thread[vert.x-eventloop-thread-<span class="number">9</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">4</span>:Thread[vert.x-eventloop-thread-<span class="number">4</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">11</span>:Thread[vert.x-eventloop-thread-<span class="number">11</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">12</span>:Thread[vert.x-eventloop-thread-<span class="number">12</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">13</span>:Thread[vert.x-eventloop-thread-<span class="number">13</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">14</span>:Thread[vert.x-eventloop-thread-<span class="number">14</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">16</span>:Thread[vert.x-eventloop-thread-<span class="number">0</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">17</span>:Thread[vert.x-eventloop-thread-<span class="number">1</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">15</span>:Thread[vert.x-eventloop-thread-<span class="number">15</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">18</span>:Thread[vert.x-eventloop-thread-<span class="number">2</span>,<span class="number">5</span>,main]</div><div class="line"><span class="number">19</span>:Thread[vert.x-eventloop-thread-<span class="number">3</span>,<span class="number">5</span>,main]</div></pre></td></tr></table></figure>
<p>可以看到尽管每个 <code>Context</code> 对应唯一的 Event Loop 线程，而每个 Event Loop 线程却可能对应多个 <code>Context</code>。</p>
<p>Event Loop Context 会在对应的 <code>EventLoop</code> 中执行 <code>Handler</code> 进行事件的处理（IO 事件，非阻塞）。Vert.x 会保证同一个 <code>Handler</code> 会一直在同一个 Event Loop 线程中执行，这样可以简化线程模型，让开发者在写 <code>Handler</code> 的时候不需要考虑并发的问题，非常方便。</p>
<p>我们来看一下 <code>Handler</code> 是如何在 <code>EventLoop</code> 上执行的。<code>EventLoopContext</code> 中实现了 <code>executeAsync</code> 方法用于包装 <code>Handler</code> 中事件处理的逻辑并将其提交至对应的 <code>EventLoop</code> 中进行执行：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">executeAsync</span><span class="params">(Handler&lt;Void&gt; task)</span> </span>&#123;</div><div class="line">  <span class="comment">// No metrics, we are on the event loop.</span></div><div class="line">  nettyEventLoop().execute(wrapTask(<span class="keyword">null</span>, task, <span class="keyword">true</span>, <span class="keyword">null</span>));</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里 Vert.x 使用了 <code>wrapTask</code> 方法将 <code>Handler</code> 封装成了一个 <code>Runnable</code> 用于向 <code>EventLoop</code> 中提交。代码比较直观，大致就是检查当前线程是否为 Vert.x 线程，然后记录事件处理开始的时间，给当前的 Vert.x 线程设置 <code>Context</code>，并且调用 <code>Handler</code> 里面的事件处理方法。具体请参考源码，这里就不贴出来了。</p>
<p>那么把封装好的 task 提交到 <code>EventLoop</code> 以后，<code>EventLoop</code> 是怎么处理的呢？这就需要更多的 Netty 相关的知识了。根据Netty 的模型，Event Loop 线程需要处理 IO 事件，普通事件（即我们的 <code>Handler</code>）以及定时事件（比如 Vert.x 的 <code>setTimer</code>）。Vert.x 会提供一个 <code>NETTY_IO_RATIO</code> 给Netty代表 <code>EventLoop</code> 处理 IO 事件时间占用的百分比（默认为 50，即 IO事件时间占用:非IO事件时间占用 = 1:1）。当 <code>EventLoop</code> 启动的时候，它会不断轮询 IO 事件及其它事件并进行处理：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title">run</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">for</span> (;;) &#123;</div><div class="line">        <span class="keyword">try</span> &#123;</div><div class="line">            <span class="keyword">switch</span> (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) &#123;</div><div class="line">                <span class="keyword">case</span> SelectStrategy.CONTINUE:</div><div class="line">                    <span class="keyword">continue</span>;</div><div class="line">                <span class="keyword">case</span> SelectStrategy.SELECT:</div><div class="line">                    select(wakenUp.getAndSet(<span class="keyword">false</span>));</div><div class="line"></div><div class="line">                    <span class="keyword">if</span> (wakenUp.get()) &#123;</div><div class="line">                        selector.wakeup();</div><div class="line">                    &#125;</div><div class="line">                <span class="keyword">default</span>:</div><div class="line">                    <span class="comment">// fallthrough</span></div><div class="line">            &#125;</div><div class="line"></div><div class="line">            cancelledKeys = <span class="number">0</span>;</div><div class="line">            needsToSelectAgain = <span class="keyword">false</span>;</div><div class="line">            <span class="keyword">final</span> <span class="keyword">int</span> ioRatio = <span class="keyword">this</span>.ioRatio;</div><div class="line">            <span class="keyword">if</span> (ioRatio == <span class="number">100</span>) &#123;</div><div class="line">                processSelectedKeys();</div><div class="line">                runAllTasks();</div><div class="line">            &#125; <span class="keyword">else</span> &#123;</div><div class="line">                <span class="keyword">final</span> <span class="keyword">long</span> ioStartTime = System.nanoTime();</div><div class="line"></div><div class="line">                processSelectedKeys();</div><div class="line"></div><div class="line">                <span class="keyword">final</span> <span class="keyword">long</span> ioTime = System.nanoTime() - ioStartTime;</div><div class="line">                runAllTasks(ioTime * (<span class="number">100</span> - ioRatio) / ioRatio);</div><div class="line">            &#125;</div><div class="line"></div><div class="line">            <span class="keyword">if</span> (isShuttingDown()) &#123;</div><div class="line">                closeAll();</div><div class="line">                <span class="keyword">if</span> (confirmShutdown()) &#123;</div><div class="line">                    <span class="keyword">break</span>;</div><div class="line">                &#125;</div><div class="line">            &#125;</div><div class="line">        &#125; <span class="keyword">catch</span> (Throwable t) &#123;</div><div class="line">            <span class="comment">// process the error</span></div><div class="line">            <span class="comment">// ...</span></div><div class="line">        &#125;</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里面 Netty 会调用 <code>processSelectedKeys</code> 方法进行 IO 事件的处理，并且会计算出处理 IO 时间所用的事件然后计算出给非 IO 事件处理分配的时间，然后调用 <code>runAllTasks</code> 方法执行所有的非 IO 任务（这里面就有我们的各个 <code>Handler</code>）。</p>
<p><code>runAllTasks</code> 会按顺序从内部的任务队列中取出任务(<code>Runnable</code>)然后进行安全执行。而我们刚才调用的 <code>NioEventLoop</code> 的 <code>execute</code> 方法其实就是将包装好的 <code>Handler</code> 置入 <code>NioEventLoop</code> 内部的任务队列中等待执行。</p>
<h2 id="Worker_Context">Worker Context</h2><p>顾名思义，Worker Context 用于跑阻塞任务。与 Event Loop Context 相似，每一个 <code>Handler</code> 都只会跑在固定的 Worker 线程下。</p>
<p>Vert.x 还提供一种 Multi-threaded worker context 可以在多个 Worker 线程下并发执行任务，这样就会出现并发问题，需要开发者自行解决并发问题。因此一般情况下我们用不到 Multi-threaded worker context。</p>
<h1 id="Verticle">Verticle</h1><p>我们再来讨论一下 <code>Verticle</code> 中的 <code>Context</code>。在部署 <code>Verticle</code> 的时候，Vert.x 会根据配置来创建 <code>Context</code> 并绑定到 Verticle 上，此后此 Verticle 上所有绑定的 <code>Handler</code> 都会在此 <code>Context</code> 上执行。相关实现位于 <code>doDeploy</code> 方法，这里摘取核心部分：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">for</span> (Verticle verticle: verticles) &#123;</div><div class="line">  WorkerExecutorImpl workerExec = poolName != <span class="keyword">null</span> ? vertx.createSharedWorkerExecutor(poolName, options.getWorkerPoolSize()) : <span class="keyword">null</span>;</div><div class="line">  WorkerPool pool = workerExec != <span class="keyword">null</span> ? workerExec.getPool() : <span class="keyword">null</span>;</div><div class="line">  <span class="comment">// 根据配置创建Context</span></div><div class="line">  ContextImpl context = options.isWorker() ? vertx.createWorkerContext(options.isMultiThreaded(), deploymentID, pool, conf, tccl) :</div><div class="line">    vertx.createEventLoopContext(deploymentID, pool, conf, tccl);</div><div class="line">  <span class="keyword">if</span> (workerExec != <span class="keyword">null</span>) &#123;</div><div class="line">    context.addCloseHook(workerExec);</div><div class="line">  &#125;</div><div class="line">  context.setDeployment(deployment);</div><div class="line">  deployment.addVerticle(<span class="keyword">new</span> VerticleHolder(verticle, context));</div><div class="line">  <span class="comment">// 此Verticle上的Handler都会在创建的context作用域内执行</span></div><div class="line">  context.runOnContext(v -&gt; &#123;</div><div class="line">    <span class="keyword">try</span> &#123;</div><div class="line">      verticle.init(vertx, context);</div><div class="line">      Future&lt;Void&gt; startFuture = Future.future();</div><div class="line">      <span class="comment">// 大家熟悉的start方法的执行点</span></div><div class="line">      verticle.start(startFuture);</div><div class="line">      startFuture.setHandler(ar -&gt; &#123;</div><div class="line">        <span class="keyword">if</span> (ar.succeeded()) &#123;</div><div class="line">          <span class="keyword">if</span> (parent != <span class="keyword">null</span>) &#123;</div><div class="line">            parent.addChild(deployment);</div><div class="line">            deployment.child = <span class="keyword">true</span>;</div><div class="line">          &#125;</div><div class="line">          vertx.metricsSPI().verticleDeployed(verticle);</div><div class="line">          deployments.put(deploymentID, deployment);</div><div class="line">          <span class="keyword">if</span> (deployCount.incrementAndGet() == verticles.length) &#123;</div><div class="line">            reportSuccess(deploymentID, callingContext, completionHandler);</div><div class="line">          &#125;</div><div class="line">        &#125; <span class="keyword">else</span> <span class="keyword">if</span> (!failureReported.get()) &#123;</div><div class="line">          reportFailure(ar.cause(), callingContext, completionHandler);</div><div class="line">        &#125;</div><div class="line">      &#125;);</div><div class="line">    &#125; <span class="keyword">catch</span> (Throwable t) &#123;</div><div class="line">      reportFailure(t, callingContext, completionHandler);</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>通过这样一种方式，Vert.x保证了<code>Verticle</code>的线程安全 —— 即某个<code>Verticle</code>上的所有<code>Handler</code>都会在同一个Vert.x线程上执行，这样也保证了<code>Verticle</code>内部成员的安全（没有race condition问题）。比如下面Verticle中处理IO及事件的处理都一直是在同一个Vert.x线程下执行的，每次打印出的线程名称应该是一样的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">TcpClientVerticle</span> <span class="keyword">extends</span> <span class="title">AbstractVerticle</span> </span>&#123;</div><div class="line">  <span class="keyword">int</span> i = <span class="number">0</span>;</div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">start</span><span class="params">()</span> <span class="keyword">throws</span> Exception </span>&#123;</div><div class="line">    vertx.createNetClient().connect(<span class="number">6666</span>, <span class="string">"localhost"</span>, ar -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (ar.succeeded()) &#123;</div><div class="line">        NetSocket socket = ar.result();</div><div class="line">        System.out.println(Thread.currentThread().getName());</div><div class="line">        socket.handler(buffer -&gt; &#123;</div><div class="line">          i++;</div><div class="line">          System.out.println(Thread.currentThread().getName());</div><div class="line">          System.out.println(<span class="string">"Net client receiving: "</span> + buffer.toString(<span class="string">"UTF-8"</span>));</div><div class="line">        &#125;);</div><div class="line">        socket.write(<span class="string">"+1s\n"</span>);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        ar.cause().printStackTrace();</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<h1 id="线程池">线程池</h1><h2 id="Event_Loop_线程池">Event Loop 线程池</h2><p>之前我们已经提到过，Event Loop 线程池的类型为 Netty 中的<code>NioEventLoopGroup</code>，里面的线程通过 Vert.x 自己的线程工厂<code>VertxThreadFactory</code>进行创建：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">eventLoopThreadFactory = <span class="keyword">new</span> VertxThreadFactory(<span class="string">"vert.x-eventloop-thread-"</span>, checker, <span class="keyword">false</span>, options.getMaxEventLoopExecuteTime());</div><div class="line">eventLoopGroup = <span class="keyword">new</span> NioEventLoopGroup(options.getEventLoopPoolSize(), eventLoopThreadFactory);</div><div class="line">eventLoopGroup.setIoRatio(NETTY_IO_RATIO);</div></pre></td></tr></table></figure>
<p>其中 Event Loop 线程的数目可以在配置中指定。</p>
<h2 id="Worker_线程池">Worker 线程池</h2><p>在之前讲 <code>executeBlocking</code> 底层实现的文章中我们已经提到过 Worker 线程池，它其实就是一种 Fixed Thread Pool:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">ExecutorService workerExec = Executors.newFixedThreadPool(options.getWorkerPoolSize(),</div><div class="line">    <span class="keyword">new</span> VertxThreadFactory(<span class="string">"vert.x-worker-thread-"</span>, checker, <span class="keyword">true</span>, options.getMaxWorkerExecuteTime()));</div><div class="line">PoolMetrics workerPoolMetrics = isMetricsEnabled() ? metrics.createMetrics(workerExec, <span class="string">"worker"</span>, <span class="string">"vert.x-worker-thread"</span>, options.getWorkerPoolSize()) : <span class="keyword">null</span>;</div><div class="line"></div><div class="line">workerPool = <span class="keyword">new</span> WorkerPool(workerExec, workerPoolMetrics);</div></pre></td></tr></table></figure>
<p>Worker线程同样由<code>VertxThreadFactory</code>构造，类型为<code>VertxThread</code>，用于执行阻塞任务。我们同样可以在配置中指定其数目。</p>
<h2 id="内部阻塞线程池">内部阻塞线程池</h2><figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">ExecutorService internalBlockingExec = Executors.newFixedThreadPool(options.getInternalBlockingPoolSize(),</div><div class="line">    <span class="keyword">new</span> VertxThreadFactory(<span class="string">"vert.x-internal-blocking-"</span>, checker, <span class="keyword">true</span>, options.getMaxWorkerExecuteTime()));</div><div class="line">PoolMetrics internalBlockingPoolMetrics = isMetricsEnabled() ? metrics.createMetrics(internalBlockingExec, <span class="string">"worker"</span>, <span class="string">"vert.x-internal-blocking"</span>, options.getInternalBlockingPoolSize()) : <span class="keyword">null</span>;</div><div class="line">internalBlockingPool = <span class="keyword">new</span> WorkerPool(internalBlockingExec, internalBlockingPoolMetrics);</div></pre></td></tr></table></figure>
<p>Internal Blocking Pool可能设计用于内部使用，在<code>executeBlocking(Action&lt;T&gt; action, Handler&lt;AsyncResult&lt;T&gt;&gt; resultHandler)</code>这个版本的方法中就使用了它。</p>
<h2 id="Acceptor_Event_Loop_线程池">Acceptor Event Loop 线程池</h2><p>大家可能会发现<code>VertxImpl</code>类中还有一个<code>acceptorEventLoopGroup</code>。顾名思义，它是Netty中的Acceptor线程池，负责处理客户端的连接请求：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">acceptorEventLoopGroup = <span class="keyword">new</span> NioEventLoopGroup(<span class="number">1</span>, acceptorEventLoopThreadFactory);</div><div class="line">acceptorEventLoopGroup.setIoRatio(<span class="number">100</span>);</div></pre></td></tr></table></figure>
<p>由于系统只有一个服务端端口需要监听，因此这里只需要一个线程。</p>
<p>Vert.x中的<code>HttpServer</code>就利用了<code>acceptorEventLoopGroup</code>处理客户端的连接请求，具体的实现后边会另起一篇介绍。</p>
]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;线程模型概述&quot;&gt;线程模型概述&lt;/h1&gt;&lt;p&gt;Vert.x 的线程模型设计的非常巧妙。总的来说，Vert.x 中主要有两种线程：&lt;strong&gt;Event Loop 线程&lt;/strong&gt; 和 &lt;strong&gt;Worker 线程&lt;/strong&gt;。其中，Event 
    
    </summary>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/categories/Vert-x/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="并发" scheme="http://www.sczyh30.com/tags/%E5%B9%B6%E5%8F%91/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="源码分析" scheme="http://www.sczyh30.com/tags/%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90/"/>
    
      <category term="Event Loop" scheme="http://www.sczyh30.com/tags/Event-Loop/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x 技术内幕 | executeBlocking 实现原理</title>
    <link href="http://www.sczyh30.com/posts/Vert-x/vertx-advanced-execute-blocking-internal/"/>
    <id>http://www.sczyh30.com/posts/Vert-x/vertx-advanced-execute-blocking-internal/</id>
    <published>2016-09-08T16:00:00.000Z</published>
    <updated>2016-11-24T06:07:28.000Z</updated>
    
    <content type="html"><![CDATA[<h1 id="引入">引入</h1><p>大家都知道，Vert.x中的<code>executeBlocking</code>方法用于执行阻塞任务，并且有两种模式：有序执行和无序执行。下面我们来看两段代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div></pre></td><td class="code"><pre><div class="line">vertx.setPeriodic(<span class="number">1000</span>, t -&gt; &#123;</div><div class="line">  vertx.executeBlocking(future -&gt; &#123;</div><div class="line">    <span class="keyword">try</span> &#123;</div><div class="line">      Thread.sleep(<span class="number">200</span>);</div><div class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</div><div class="line">      e.printStackTrace();</div><div class="line">    &#125;</div><div class="line">    System.out.println(Thread.currentThread().getName());</div><div class="line">    future.complete();</div><div class="line">  &#125;, r -&gt; &#123;&#125;);</div><div class="line">&#125;);</div><div class="line"></div><div class="line">vertx.setPeriodic(<span class="number">1000</span>, t -&gt; &#123;</div><div class="line">  vertx.executeBlocking(future -&gt; &#123;</div><div class="line">    <span class="keyword">try</span> &#123;</div><div class="line">      Thread.sleep(<span class="number">2000</span>);</div><div class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</div><div class="line">      e.printStackTrace();</div><div class="line">    &#125;</div><div class="line">    System.out.println(Thread.currentThread().getName());</div><div class="line">    future.complete();</div><div class="line">  &#125;, r -&gt; &#123;&#125;);</div><div class="line">&#125;);</div></pre></td></tr></table></figure>
<p>我们思考一下，每段代码每次执行的时候使用的线程相同么？正常情况下大家都知道<code>executeBlocking</code>底层使用了Worker线程池，因此貌似两种情况没什么区别，都是轮询Worker线程池，每次可能用不同的Worker线程。但是我们测一下：</p>
<p>第一段代码：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">vert.x-worker-thread-0</div><div class="line">vert.x-worker-thread-1</div><div class="line">vert.x-worker-thread-2</div><div class="line">vert.x-worker-thread-3</div><div class="line">vert.x-worker-thread-4</div></pre></td></tr></table></figure>
<p>第二段代码：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">vert.x-worker-thread-0</div><div class="line">vert.x-worker-thread-0</div><div class="line">vert.x-worker-thread-0</div><div class="line">vert.x-worker-thread-0</div><div class="line">vert.x-worker-thread-0</div></pre></td></tr></table></figure>
<p>额。。。两段代码每次执行的线程居然有差异？第二次为什么每次都用相同的Worker线程？其实，大家可能忽略了一点：<code>executeBlocking</code>方法默认<strong>顺序执行</strong>提交的阻塞任务。今天我们就来探究一下<code>executeBlocking</code>内部的实现。</p>
<h1 id="Worker线程池">Worker线程池</h1><p>我们来回顾一下Vert.x底层的Worker线程池，它在创建<code>VertxImpl</code>实例的时候进行初始化：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">ExecutorService workerExec = Executors.newFixedThreadPool(options.getWorkerPoolSize(),</div><div class="line">    <span class="keyword">new</span> VertxThreadFactory(<span class="string">"vert.x-worker-thread-"</span>, checker, <span class="keyword">true</span>, options.getMaxWorkerExecuteTime()));</div><div class="line">PoolMetrics workerPoolMetrics = isMetricsEnabled() ? metrics.createMetrics(workerExec, <span class="string">"worker"</span>, <span class="string">"vert.x-worker-thread"</span>, options.getWorkerPoolSize()) : <span class="keyword">null</span>;</div><div class="line"></div><div class="line">workerPool = <span class="keyword">new</span> WorkerPool(workerExec, workerPoolMetrics);</div></pre></td></tr></table></figure>
<p>可以看到底层的Worker线程池本质上是一种<code>FixedThreadPool</code>，里面的线程由<code>VertxThreadFactory</code>控制生成，对应的线程类型为<code>VertxThread</code>。Vert.x内部用<code>WorkerPool</code>类对线程池以及线程池相关的Metrics类进行了封装。</p>
<h1 id="阻塞任务在哪里执行？">阻塞任务在哪里执行？</h1><p>有了Worker线程池的基础，我们来看一下<code>Vertx</code>实例中的<code>executeBlocking</code>方法，它的过程很简单：获取当前的Vert.x Context（没有就创建），然后委托调用<code>Context</code>里的<code>executeBlocking</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">executeBlocking</span><span class="params">(Handler&lt;Future&lt;T&gt;&gt; blockingCodeHandler, <span class="keyword">boolean</span> ordered,</span></span></div><div class="line">                                Handler&lt;AsyncResult&lt;T&gt;&gt; asyncResultHandler) &#123;</div><div class="line">  ContextImpl context = getOrCreateContext();</div><div class="line">  context.executeBlocking(blockingCodeHandler, ordered, asyncResultHandler);</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">executeBlocking</span><span class="params">(Handler&lt;Future&lt;T&gt;&gt; blockingCodeHandler,</span></span></div><div class="line">                                Handler&lt;AsyncResult&lt;T&gt;&gt; asyncResultHandler) &#123;</div><div class="line">  executeBlocking(blockingCodeHandler, <span class="keyword">true</span>, asyncResultHandler);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在此方法中可以看到，<code>ordered</code>标志位默认为<code>true</code>，即默认按提交的次序执行阻塞任务。</p>
<p>我们再来看一下<code>ContextImpl</code>类中的<code>executeBlocking</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">executeBlocking</span><span class="params">(Handler&lt;Future&lt;T&gt;&gt; blockingCodeHandler, <span class="keyword">boolean</span> ordered, Handler&lt;AsyncResult&lt;T&gt;&gt; resultHandler)</span> </span>&#123;</div><div class="line">  executeBlocking(<span class="keyword">null</span>, blockingCodeHandler, resultHandler, ordered ? workerExec : workerPool.executor(), workerPool.metrics());</div><div class="line">&#125;</div><div class="line"></div><div class="line">&lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">executeBlocking</span><span class="params">(Action&lt;T&gt; action, Handler&lt;Future&lt;T&gt;&gt; blockingCodeHandler,</span></span></div><div class="line">    Handler&lt;AsyncResult&lt;T&gt;&gt; resultHandler,</div><div class="line">    Executor exec, PoolMetrics metrics) &#123;</div><div class="line">  Object queueMetric = metrics != <span class="keyword">null</span> ? metrics.submitted() : <span class="keyword">null</span>;</div><div class="line">  <span class="keyword">try</span> &#123;</div><div class="line">    exec.execute(() -&gt; &#123;</div><div class="line">      Object execMetric = <span class="keyword">null</span>;</div><div class="line">      <span class="keyword">if</span> (metrics != <span class="keyword">null</span>) &#123;</div><div class="line">        execMetric = metrics.begin(queueMetric);</div><div class="line">      &#125;</div><div class="line">      Future&lt;T&gt; res = Future.future();</div><div class="line">      <span class="keyword">try</span> &#123;</div><div class="line">        <span class="keyword">if</span> (blockingCodeHandler != <span class="keyword">null</span>) &#123;</div><div class="line">          ContextImpl.setContext(<span class="keyword">this</span>);</div><div class="line">          blockingCodeHandler.handle(res);</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">          T result = action.perform();</div><div class="line">          res.complete(result);</div><div class="line">        &#125;</div><div class="line">      &#125; <span class="keyword">catch</span> (Throwable e) &#123;</div><div class="line">        res.fail(e);</div><div class="line">      &#125;</div><div class="line">      <span class="keyword">if</span> (metrics != <span class="keyword">null</span>) &#123;</div><div class="line">        metrics.end(execMetric, res.succeeded());</div><div class="line">      &#125;</div><div class="line">      <span class="keyword">if</span> (resultHandler != <span class="keyword">null</span>) &#123;</div><div class="line">        runOnContext(v -&gt; res.setHandler(resultHandler));</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">  &#125; <span class="keyword">catch</span> (RejectedExecutionException ignore) &#123;</div><div class="line">    <span class="comment">// Pool is already shut down</span></div><div class="line">    <span class="keyword">if</span> (metrics != <span class="keyword">null</span>) &#123;</div><div class="line">      metrics.rejected(queueMetric);</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>它调用了另一个具体版本的<code>executeBlocking</code>方法，其中第四个参数即为要执行阻塞任务的线程池。如果要有序执行(<code>ordered</code>为true)，底层就使用<code>context</code>实例里的<code>workerExec</code>线程池；如果无序执行，就调用<code>workerPool</code>的<code>executor</code>方法获取另一种线程池。</p>
<p>看到这里，我们大致已经想到了，有序执行和无序执行两种模式使用不同的线程池，因此底层实现肯定有差异。我们来看一下前面提到的两个线程池，它们都是<code>ContextImpl</code>类的成员变量：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> <span class="keyword">final</span> WorkerPool workerPool;</div><div class="line"><span class="keyword">protected</span> <span class="keyword">final</span> Executor workerExec;</div></pre></td></tr></table></figure>
<p>在通过<code>Vertx</code>实例创建<code>Context</code>的时候，这几个变量会被初始化，其来源就是之前我们看过的<code>VertxImpl</code>实例中的Worker线程池。看一下<code>ContextImpl</code>类的构造函数就一目了然了：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">this</span>.workerPool = workerPool;</div><div class="line"><span class="keyword">this</span>.workerExec = workerPool.createOrderedExecutor();</div></pre></td></tr></table></figure>
<p>嗯。。。有序执行对应的线程池通过<code>workerPool</code>的<code>createOrderedExecutor</code>方法获得，而无序执行对应的线程池通过<code>workerPool</code>的<code>executor</code>方法获得。因此，<code>WorkerPool</code>类是一个关键点，我们稍后就看一下其实现。</p>
<p>注意Vert.x规定,<code>blockingCodeHandler</code>中的逻辑（即阻塞任务）在Worker线程内执行，而<code>resultHandler</code>内的逻辑（结果处理）需要在Vert.x Conext中执行，因此前面需要预先设置当前使用的Worker线程的<code>Context</code>为<code>this</code>以便后面调用<code>runOnContext</code>方法执行结果处理逻辑。</p>
<p>下面就来看一下有序执行和无序执行这两种线程池的具体区别。</p>
<h1 id="无序执行">无序执行</h1><p>我们看一下<code>WorkerPool</code>类的源码中获取无序执行线程池的逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function">ExecutorService <span class="title">executor</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">return</span> pool;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到<code>executor</code>方法直接返回了内部的<code>pool</code>线程池，而<code>pool</code>线程池其实就是<code>VertxImpl</code>中的<code>workerExec</code>线程池：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">workerPool = <span class="keyword">new</span> WorkerPool(workerExec, workerPoolMetrics);</div></pre></td></tr></table></figure>
<p>OK！如果大家熟悉并发的话，大家应该对无序执行对应的线程池 —— Worker线程池的行为非常清楚了。它属于一种<code>FixedThreadPool</code>，底层通过阻塞队列<code>LinkedBlockingQueue</code>实现。底层通过轮询算法获取Worker线程执行任务。</p>
<h1 id="有序执行">有序执行</h1><p>下面是时候看有序执行对应的逻辑了：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">final</span> OrderedExecutorFactory orderedFact;</div><div class="line"><span class="keyword">private</span> <span class="keyword">final</span> ExecutorService pool;</div><div class="line"><span class="keyword">private</span> <span class="keyword">final</span> PoolMetrics metrics;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="title">WorkerPool</span><span class="params">(ExecutorService pool, PoolMetrics metrics)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.orderedFact = <span class="keyword">new</span> OrderedExecutorFactory(pool);</div><div class="line">  <span class="keyword">this</span>.pool = pool;</div><div class="line">  <span class="keyword">this</span>.metrics = metrics;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function">Executor <span class="title">createOrderedExecutor</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">return</span> orderedFact.getExecutor();</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到有序执行对应的线程池是通过<code>OrderedExecutorFactory</code>创建的。其实，<code>OrderedExecutorFactory</code>类会生成真正的有序执行线程池<code>OrderedExecutor</code>，它其实是对Worker线程池<code>pool</code>的一个简单包装，仅仅添加了有序执行相关的逻辑，最后还是委托Worker线程池进行任务处理。</p>
<p>那么<code>OrderedExecutor</code>是如何实现顺序执行的呢？<code>OrderedExecutor</code>内部维护着一个任务队列。每当调用<code>executeBlocking</code>方法执行阻塞过程的时候，Vert.x会将阻塞过程包装成<code>Runnable</code>然后置入<code>OrderedExecutor</code>中的任务队列中；同时如果<code>OrderedExecutor</code>没有开始执行任务，就委托内部的Worker线程池执行任务：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">execute</span><span class="params">(Runnable command)</span> </span>&#123;</div><div class="line">  <span class="keyword">synchronized</span> (tasks) &#123;</div><div class="line">    tasks.add(command);</div><div class="line">    <span class="keyword">if</span> (!running) &#123;</div><div class="line">      running = <span class="keyword">true</span>;</div><div class="line">      parent.execute(runner);</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>从代码中可以看出，最后委托Worker线程池执行的线程其实是又包装了一层的<code>runner</code>线程。<code>runner</code>的逻辑不难想：不断地从任务队列中取出队首的<code>Runnable</code>然后调用其<code>run</code>方法执行（相当于执行了此任务，只不过在runner对应的线程中）；如果没有任务了就结束本线程。</p>
<p>这里就出现了一种情况：大批量提交阻塞任务的时候，线程池的状态<code>running</code>一直为<code>true</code>，此时所有的任务都积压到任务队列中，而执行所有任务的线程只有一个 —— <code>runner</code>对应的线程。这种情况其实很好想，因为要保证有序执行，就只能让它一个接一个地在同个线程中执行。如果在不同线程中依次执行则不好调度，如果直接并行执行则不能保证有序性。</p>
<p>所以，根据<code>OrderedExecutor</code>线程池的内部实现，只要提交任务的间隔时间小于任务执行的时间，底层其实就仅执行了一次<code>runner</code>，也就是说所有提交的阻塞任务都只在一个线程下跑（running标志位控制）。</p>
<p>这样就可以很好地解释我们一开始提出的问题了。当<code>sleep(200), setPeriodic(1000)</code>的时候，提交任务的间隔时间大于任务执行的时间，这样每次的<code>runner</code>就可以在下一个任务提交之前执行完，因此每次所用的线程会不同（轮询策略）；而<code>sleep(2000), setPeriodic(1000)</code>的时候，提交任务的间隔时间小于任务执行的时间，底层最后都归结到一个<code>runner</code>中执行了，因此所有过程都是在同一个Worker线程执行的（很好想，保证有序就要串行执行）。</p>
<p>当然，如果不想有序执行，可以用<code>void executeBlocking(Handler&lt;Future&lt;T&gt;&gt; blockingCodeHandler, boolean ordered, Handler&lt;AsyncResult&lt;T&gt;&gt; asyncResultHandler)</code>这个版本的<code>executeBlocking</code>方法，并将<code>ordered</code>标志位设为<code>false</code>。根据上面的源码，底层会直接使用Worker线程池而不是<code>OrderedExecutor</code>线程池，这样就不会有上面<code>OrderedExecutor</code>的情况了。</p>
]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;引入&quot;&gt;引入&lt;/h1&gt;&lt;p&gt;大家都知道，Vert.x中的&lt;code&gt;executeBlocking&lt;/code&gt;方法用于执行阻塞任务，并且有两种模式：有序执行和无序执行。下面我们来看两段代码：&lt;/p&gt;
&lt;figure class=&quot;highlight java&quot;&gt;
    
    </summary>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/categories/Vert-x/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="并发" scheme="http://www.sczyh30.com/tags/%E5%B9%B6%E5%8F%91/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
      <category term="源码分析" scheme="http://www.sczyh30.com/tags/%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x 技术内幕 | 异步 RPC 实现原理</title>
    <link href="http://www.sczyh30.com/posts/Vert-x/vertx-advanced-async-rpc/"/>
    <id>http://www.sczyh30.com/posts/Vert-x/vertx-advanced-async-rpc/</id>
    <published>2016-09-06T16:00:00.000Z</published>
    <updated>2018-09-23T14:15:44.000Z</updated>
    
    <content type="html"><![CDATA[<p>经常有一些开发者在 group 中问到，如何利用 Vert.x 进行 RPC 通信。其实，Vert.x 提供了一个组件 —— <strong>Vert.x Service Proxy</strong>，专门用于进行异步 RPC 通信（通过 Event Bus）。Vert.x Service Proxy 会自动生成代理类进行消息的包装与解码、发送与接收以及超时处理，可以为我们省掉不少代码。之前我在 Vert.x Blueprint 中已经详细讲解了 <strong>Vert.x Service Proxy</strong> 的使用，大家可以参考 <a href="http://www.sczyh30.com/vertx-blueprint-job-queue/cn/kue-core/index.html#异步rpc">Vert.x Kue 文档</a> 中的相关部分。本篇文章中我们将探索一下通过 <strong>Vert.x Service Proxy</strong> 生成的代理类进行异步RPC的原理，对应的 Vert.x 版本为 <strong>3.3.2</strong>。</p>
<p>传统的 RPC 想必大家都不陌生，但是传统的 RPC 有个缺陷：传统的 RPC 都是阻塞型的，当调用者远程调用服务时需要阻塞着等待调用结果，这与 Vert.x 的异步开发模式相违背；而且，传统的 RPC 未对容错而设计。</p>
<p>因此，Vert.x 提供了 Service Proxy 用于进行异步 RPC，其底层依托 Clustered Event Bus 进行通信。我们只需要按照规范编写我们的服务接口（一般称为 Event Bus 服务），并加上 <code>@ProxyGen</code> 注解，Vert.x 就会自动为我们生成相应的代理类在底层处理 RPC。有了 Service Proxy，我们只需给异步方法提供一个回调函数 <code>Handler&lt;AsyncResult&lt;T&gt;&gt;</code>，在调用结果发送过来的时候会自动调用绑定的回调函数进行相关的处理，这样就与 Vert.x 的异步开发模式相符了。由于 <code>AsyncResult</code> 本身就是为容错而设计的（两个状态），因此这里的 RPC 也具有了容错性。</p>
<h1 id="原理简介">原理简介</h1><p>假设有一个 Event Bus 服务接口：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@ProxyGen</span></div><div class="line"><span class="meta">@VertxGen</span></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">SomeService</span> </span>&#123;</div><div class="line"></div><div class="line">  String SERVICE_ADDRESS = <span class="string">"service.example"</span>;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">static</span> SomeService <span class="title">createService</span><span class="params">(Vertx vertx, JsonObject config)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">new</span> SomeServiceImpl(vertx, config);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">static</span> SomeService <span class="title">createProxy</span><span class="params">(Vertx vertx)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> ProxyHelper.createProxy(SomeService.class, vertx, SERVICE_ADDRESS);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">SomeService <span class="title">process</span><span class="params">(String id, Handler&lt;AsyncResult&lt;JsonObject&gt;&gt; resultHandler)</span></span>;</div><div class="line"></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里定义了一个异步方法 <code>process</code>，其异步调用返回的结果是 <code>AsyncResult&lt;JsonObject&gt;</code> 类型的。由于异步 RPC 底层通过 Clustered Event Bus 进行通信，我们需要给器指定一个通信地址 <code>SERVICE_ADDRESS</code>。<code>@Fluent</code> 注解代表此方法返回自身，便于进行组合。我们同时还提供了两个辅助方法：<code>createService</code> 方法用于创建服务实例，而 <code>createProxy</code> 方法则通过 <code>ProxyHelper</code> 辅助类创建服务代理实例。</p>
<p>假设服务提供端A注册了一个 <code>SomeService</code> 类型的服务代理，服务调用端B需要通过异步RPC调用服务的 <code>process</code> 方法，此时调用端 B 可以利用 <code>ProxyHelper</code> 获取服务实例并进行服务调用。B 中获取的服务其实是一个 <strong>服务代理类</strong>，而真正的服务实例在 A 处。何为服务代理？服务代理可以帮助我们向服务提供端发送调用请求，并且响应调用结果。那么如何发送调用请求呢？相信大家能想到，是调用端B将调用参数和方法名称等必要信息包装成集群消息(<code>ClusteredMessage</code>)，然后通过 <code>send</code> 方法将请求通过 Clustered Event Bus 发送至服务提供端A处（需要提供此服务的通信地址）。A 在注册服务的时候会创建一个 <code>MessageConsumer</code> 监听此服务的地址来响应调用请求。当接收到调用请求的时候，A 会在本地调用方法，并将结果回复至调用端。所以异步RPC本质上其实是一个基于 <strong>代理模式</strong> 的 <strong>Request/Response</strong> 消息模式。</p>
<p>用时序图来描述一下上述过程：</p>
<p><img src="/assets/blog-img-k1/vertx-async-rpc-sequence.png" alt="Sequence Diagram of Async RPC"></p>
<h1 id="引入">引入</h1><p>以之前的 <code>SomeService</code> 接口为例，我们可以在集群中的一个节点上注册服务实例：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">SomeService service = SomeService.createService(vertx, config);</div><div class="line">ProxyHelper.registerService(SomeService.class, vertx, service, SomeService.SERVICE_ADDRESS);</div></pre></td></tr></table></figure>
<p>然后在另一个节点上获取此服务实例的代理，并进行服务调用。调用的时候看起来就像在本地调用(LPC)一样，其实是进行了 RPC 通信：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line">SomeService proxyService = SomeService.createProxy(vertx);</div><div class="line"></div><div class="line"><span class="comment">// invoke the service</span></div><div class="line">proxyService.process(<span class="string">"fuck"</span>, ar -&gt; &#123;</div><div class="line">  <span class="comment">// process the result...</span></div><div class="line">&#125;);</div></pre></td></tr></table></figure>
<p>其实，这里获取到的 <code>proxyService</code> 实例的真正类型是 Vert.x 自动生成的服务代理类 <code>SomeServiceVertxEBProxy</code> 类，里面封装了通过 Event Bus 进行通信的逻辑。我们首先来讲一下 Service Proxy 生成代理类的命名规范。</p>
<h1 id="代理类命名规范">代理类命名规范</h1><p>Vert.x Service Proxy 在生成代理类时遵循一定的规范。假设有一 Event Bus 服务接口<code>SomeService</code>，Vert.x 会自动为其生成代理类以及代理处理器：</p>
<ul>
<li>代理类的命名规范为 <strong>接口名</strong> + <code>VertxEBProxy</code>。比如<code>SomeService</code>接口对应的代理类名称为<code>SomeServiceVertxEBProxy</code></li>
<li>代理类会继承原始的服务接口并实现所有方法的代理逻辑</li>
<li>代理处理器的命名规范为 <strong>接口名</strong> + <code>VertxProxyHandler</code>。比如<code>SomeService</code>接口对应的代理处理器名称为<code>SomeServiceVertxProxyHandler</code></li>
<li>代理处理器会继承<code>ProxyHandler</code>抽象类</li>
</ul>
<p><code>ProxyHelper</code>辅助类中注册服务以及创建代理都是遵循了这个规范。</p>
<h1 id="在Event_Bus上注册服务">在Event Bus上注册服务</h1><p>我们通过<code>ProxyHelper</code>辅助类中的<code>registerService</code>方法来向Event Bus上注册Event Bus服务，来看其具体实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="keyword">static</span> &lt;T&gt; <span class="function">MessageConsumer&lt;JsonObject&gt; <span class="title">registerService</span><span class="params">(Class&lt;T&gt; clazz, Vertx vertx, T service, String address,</span></span></div><div class="line">                                                              <span class="keyword">boolean</span> topLevel,</div><div class="line">                                                              <span class="keyword">long</span> timeoutSeconds) &#123;</div><div class="line">  String handlerClassName = clazz.getName() + <span class="string">"VertxProxyHandler"</span>;</div><div class="line">  Class&lt;?&gt; handlerClass = loadClass(handlerClassName, clazz);</div><div class="line">  Constructor constructor = getConstructor(handlerClass, Vertx.class, clazz, <span class="keyword">boolean</span>.class, <span class="keyword">long</span>.class);</div><div class="line">  Object instance = createInstance(constructor, vertx, service, topLevel, timeoutSeconds);</div><div class="line">  ProxyHandler handler = (ProxyHandler) instance;</div><div class="line">  <span class="keyword">return</span> handler.registerHandler(address);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先根据约定生成对应的代理<code>Handler</code>的名称，然后通过类加载器加载对应的<code>Handler</code>类，再通过反射来创建代理<code>Handler</code>的实例，最后调用<code>handler</code>的<code>registerHandler</code>方法注册服务地址。</p>
<p><code>registerHandler</code>方法的实现在Vert.x生成的各个代理处理器中。以之前的<code>SomeService</code>为例，我们来看一下其对应的代理处理器<code>SomeServiceVertxProxyHandler</code>实现。首先是注册并订阅地址的<code>registerHandler</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> MessageConsumer&lt;JsonObject&gt; <span class="title">registerHandler</span><span class="params">(String address)</span> </span>&#123;</div><div class="line">  MessageConsumer&lt;JsonObject&gt; consumer = vertx.eventBus().&lt;JsonObject&gt;consumer(address).handler(<span class="keyword">this</span>);</div><div class="line">  <span class="keyword">this</span>.setConsumer(consumer);</div><div class="line">  <span class="keyword">return</span> consumer;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>registerHandler</code>方法的实现非常简单，就是通过<code>consumer</code>方法在<code>address</code>地址上绑定了<code>SomeServiceVertxProxyHandler</code>自身。那么<code>SomeServiceVertxProxyHandler</code>是如何处理来自服务调用端的服务调用请求，并将调用结果返回到请求端呢？在回答这个问题之前，我们先来看看代理端（调用端）是如何发送服务调用请求的，这就要看对应的服务代理类的实现了。</p>
<h1 id="服务调用">服务调用</h1><p>我们来看一下服务调用端是如何发出服务调用请求的消息的。之前已经介绍过，服务调用端是通过Event Bus的<code>send</code>方法发送调用请求的，并且会提供一个<code>replyHandler</code>来等待方法调用的结果。调用的方法名称会存放在消息中名为<code>action</code>的header中。以之前<code>SomeService</code>的代理类<code>SomeServiceVertxEBProxy</code>中<code>process</code>方法的请求为例：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> SomeService <span class="title">process</span><span class="params">(String id, Handler&lt;AsyncResult&lt;JsonObject&gt;&gt; resultHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (closed) &#123;</div><div class="line">    resultHandler.handle(Future.failedFuture(<span class="keyword">new</span> IllegalStateException(<span class="string">"Proxy is closed"</span>)));</div><div class="line">    <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">  &#125;</div><div class="line">  JsonObject _json = <span class="keyword">new</span> JsonObject();</div><div class="line">  _json.put(<span class="string">"id"</span>, id);</div><div class="line">  DeliveryOptions _deliveryOptions = (_options != <span class="keyword">null</span>) ? <span class="keyword">new</span> DeliveryOptions(_options) : <span class="keyword">new</span> DeliveryOptions();</div><div class="line">  _deliveryOptions.addHeader(<span class="string">"action"</span>, <span class="string">"process"</span>);</div><div class="line">  _vertx.eventBus().&lt;JsonObject&gt;send(_address, _json, _deliveryOptions, res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.failed()) &#123;</div><div class="line">      resultHandler.handle(Future.failedFuture(res.cause()));</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      resultHandler.handle(Future.succeededFuture(res.result().body()));</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到代理类把此方法传入的参数都放到一个<code>JsonObject</code>中了，并将要调用的方法名称存放在消息中名为<code>action</code>的header中。代理方法通过<code>send</code>方法将包装好的消息发送至之前注册的服务地址处，并且绑定<code>replyHandler</code>等待调用结果，然后使用我们传入到<code>process</code>方法中的<code>resultHandler</code>对结果进行处理。是不是很简单呢？</p>
<h1 id="服务提供端的调用逻辑">服务提供端的调用逻辑</h1><p>调用请求发出之后，我们的服务提供端就会收到调用请求消息，然后执行<code>SomeServiceVertxProxyHandler</code>中的处理逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">handle</span><span class="params">(Message&lt;JsonObject&gt; msg)</span> </span>&#123;</div><div class="line">  <span class="keyword">try</span> &#123;</div><div class="line">    JsonObject json = msg.body();</div><div class="line">    String action = msg.headers().get(<span class="string">"action"</span>);</div><div class="line">    <span class="keyword">if</span> (action == <span class="keyword">null</span>) &#123;</div><div class="line">      <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"action not specified"</span>);</div><div class="line">    &#125;</div><div class="line">    accessed();</div><div class="line">    <span class="keyword">switch</span> (action) &#123;</div><div class="line">      <span class="keyword">case</span> <span class="string">"process"</span>: &#123;</div><div class="line">        service.process((java.lang.String)json.getValue(<span class="string">"id"</span>), createHandler(msg));</div><div class="line">        <span class="keyword">break</span>;</div><div class="line">      &#125;</div><div class="line">      <span class="keyword">default</span>: &#123;</div><div class="line">        <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"Invalid action: "</span> + action);</div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">  &#125; <span class="keyword">catch</span> (Throwable t) &#123;</div><div class="line">    msg.reply(<span class="keyword">new</span> ServiceException(<span class="number">500</span>, t.getMessage()));</div><div class="line">    <span class="keyword">throw</span> t;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>handle</code>方法首先从消息header中获取方法名称，如果获取不到则调用失败；接着<code>handle</code>方法会调用<code>accessed</code>方法记录最后调用服务的时间戳，这是为了实现超时的逻辑，后面我们会讲。接着<code>handle</code>方法会根据方法名称分派对应的逻辑，在“真正”的服务实例上调用方法。注意异步RPC的过程本质是 <strong>Request/Response</strong> 模式，因此这里的异步结果处理函数<code>resultHandler</code>应该将调用结果发送回调用端。此<code>resultHandler</code>是通过<code>createHandler</code>方法生成的，逻辑很清晰：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> &lt;T&gt; Handler&lt;AsyncResult&lt;T&gt;&gt; createHandler(Message msg) &#123;</div><div class="line">  <span class="keyword">return</span> res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.failed()) &#123;</div><div class="line">      <span class="keyword">if</span> (res.cause() <span class="keyword">instanceof</span> ServiceException) &#123;</div><div class="line">        msg.reply(res.cause());</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        msg.reply(<span class="keyword">new</span> ServiceException(-<span class="number">1</span>, res.cause().getMessage()));</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">if</span> (res.result() != <span class="keyword">null</span>  &amp;&amp; res.result().getClass().isEnum()) &#123;</div><div class="line">        msg.reply(((Enum) res.result()).name());</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        msg.reply(res.result());</div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">  &#125;;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这样，一旦在服务提供端的调用过程完成时，调用结果就会被发送回调用端。这样调用端就可以调用结果执行真正的处理逻辑了。</p>
<h1 id="超时处理">超时处理</h1><p>Vert.x 自动生成的代理处理器内都封装了一个简单的超时处理逻辑，它是通过定时器定时检查最后的调用时间实现的。逻辑比较简单，直接放上相关逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="title">SomeServiceVertxProxyHandler</span><span class="params">(Vertx vertx, SomeService service, <span class="keyword">boolean</span> topLevel, <span class="keyword">long</span> timeoutSeconds)</span> </span>&#123;</div><div class="line">  <span class="comment">// 前面代码略。。。</span></div><div class="line">  <span class="keyword">if</span> (timeoutSeconds != -<span class="number">1</span> &amp;&amp; !topLevel) &#123;</div><div class="line">    <span class="keyword">long</span> period = timeoutSeconds * <span class="number">1000</span> / <span class="number">2</span>;</div><div class="line">    <span class="keyword">if</span> (period &gt; <span class="number">10000</span>) &#123;</div><div class="line">      period = <span class="number">10000</span>;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">this</span>.timerID = vertx.setPeriodic(period, <span class="keyword">this</span>::checkTimedOut);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    <span class="keyword">this</span>.timerID = -<span class="number">1</span>;</div><div class="line">  &#125;</div><div class="line">  accessed();</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">checkTimedOut</span><span class="params">(<span class="keyword">long</span> id)</span> </span>&#123;</div><div class="line">  <span class="keyword">long</span> now = System.nanoTime();</div><div class="line">  <span class="keyword">if</span> (now - lastAccessed &gt; timeoutSeconds * <span class="number">1000000000</span>) &#123;</div><div class="line">    close();</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>一旦超时，就自动调用<code>close</code>方法终止定时器，注销响应服务调用请求的consumer并关闭代理。</p>
<h1 id="代码是如何生成的？">代码是如何生成的？</h1><p>大家可能会很好奇，这些服务代理类是怎么生成出来的？其实，这都是 Vert.x Codegen 的功劳。Vert.x Codegen 的本质是一个 <strong>注解处理器</strong>(APT)，它可以扫描源码中是否包含要处理的注解，检查规范后根据响应的模板生成对应的代码，这就是注解处理器的作用(注解处理器于 JDK 1.6 引入)。为了让 Codegen 正确地生成代码，我们需要配置编译参数来确保注解处理器能够正常的工作，具体的可以参考 <a href="https://github.com/vert-x3/vertx-codegen/blob/master/README.md" target="_blank" rel="external">Vert.x Codegen 的文档</a> （之前里面缺了 Gradle 相关的实例，我给补上了）。</p>
<p>Vert.x Codegen 使用 MVEL2 作为生成代码的模板，扩展名为 <code>*.templ</code>，比如代理类和代理处理器的模板就位于 <a href="https://github.com/vert-x3/vertx-service-proxy/tree/master/src/main/resources/serviceproxy/template" target="_blank" rel="external">vert-x3/vertx-service-proxy</a> 中，配置文件类似于这样：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div></pre></td><td class="code"><pre><div class="line"></div><div class="line">&#123;</div><div class="line">  <span class="attr">"name"</span>: <span class="string">"Proxy"</span>,</div><div class="line">  <span class="attr">"generators"</span>: [</div><div class="line">    &#123;</div><div class="line">      <span class="attr">"kind"</span>: <span class="string">"proxy"</span>,</div><div class="line">      <span class="attr">"fileName"</span>: <span class="string">"ifaceFQCN + 'VertxEBProxy.java'"</span>,</div><div class="line">      <span class="attr">"templateFileName"</span>: <span class="string">"serviceproxy/template/proxygen.templ"</span></div><div class="line">    &#125;,&#123;</div><div class="line">      <span class="attr">"kind"</span>: <span class="string">"proxy"</span>,</div><div class="line">      <span class="attr">"fileName"</span>: <span class="string">"ifaceFQCN + 'VertxProxyHandler.java'"</span>,</div><div class="line">      <span class="attr">"templateFileName"</span>: <span class="string">"serviceproxy/template/handlergen.templ"</span></div><div class="line">    &#125;</div><div class="line">  ]</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>具体的代码生成逻辑还要涉及 APT 及 MVEL2 的知识，这里就不展开讲了，有兴趣的朋友可以研究研究 Vert.x Codegen 的源码。</p>
<h1 id="优点与缺点">优点与缺点</h1><p>Vert.x 提供的这种 Async RPC 有着许多优点：</p>
<ul>
<li>通过 Clustered Event Bus 传输消息，不需引入其它额外的组件；</li>
<li>自动生成代理类及代理处理器，可以帮助我们做消息封装、传输、编码解码以及超时处理等问题，省掉不少冗余代码，让我们可以以 LPC 的方式进行 RPC 通信；</li>
<li>多语言支持(Polyglot support)。这是 Vert.x 的一大亮点。只要加上 <code>@VertxGen</code> 注解并在编译期依赖中加上对应语言的依赖(如<code>vertx-lang-ruby</code>)，Vert.x Codegen 就会自动处理注解并生成对应语言的服务代理（通过调用 Java 版本的服务代理实现）。这样 Async RPC 可以真正地做到不限 language。</li>
</ul>
<p>当然 Vert.x 要求我们的服务接口必须是 <strong>基于回调的</strong>，这样写起来可能会不优雅。还好 <code>@VertxGen</code> 注解支持生成Rx版本的服务类，因此只要加上 <code>vertx-rx-java</code> 依赖，Codegen 就能生成对应的Rx风格的服务类（异步方法返回 <code>Observable</code>），这样我们就能以更 reactive 的风格来构建应用了，岂不美哉？</p>
<p>当然，为了考虑多语言支持的兼容性，Vert.x 在传递消息的时候依然使用了传统的 JSON，这样传输效率可能不如 Protobuf 高，但是不一定成为瓶颈（看业务情况；真正的瓶颈一般还是在 DB 上）。</p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;经常有一些开发者在 group 中问到，如何利用 Vert.x 进行 RPC 通信。其实，Vert.x 提供了一个组件 —— &lt;strong&gt;Vert.x Service Proxy&lt;/strong&gt;，专门用于进行异步 RPC 通信（通过 Event Bus）。Vert.x
    
    </summary>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/categories/Vert-x/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
      <category term="源码分析" scheme="http://www.sczyh30.com/tags/%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90/"/>
    
  </entry>
  
  <entry>
    <title>Distributed System | RPC 模块设计与实现</title>
    <link href="http://www.sczyh30.com/posts/Distributed-System/distributed-system-rpc-design/"/>
    <id>http://www.sczyh30.com/posts/Distributed-System/distributed-system-rpc-design/</id>
    <published>2016-09-06T16:00:00.000Z</published>
    <updated>2018-09-23T12:46:18.000Z</updated>
    
    <content type="html"><![CDATA[<p>RPC 是分布式系统中不可缺少的一部分。之前接触过几种 RPC 组件，这里就总结一下常见 RPC 模块的设计思想和实现。最后我们来设计一个可以方便进行 RPC 调用的 RPC 模块。</p>
<h1 id="RPC_模块设计需要考虑的问题">RPC 模块设计需要考虑的问题</h1><p>RPC 模块将网络通信的过程封装成了方法调用的过程。从使用者的角度来看，在调用端进行RPC调用，就像进行本地函数调用一样；而在背后，RPC 模块会将先调用端的函数名称、参数等调用信息序列化，其中序列化的方式有很多种，比如 Java 原生序列化、JSON、Protobuf 等。接着 RPC 模块会将序列化后的消息通过某种协议(如 TCP, AMQP 等)发送到被调用端，被调用端在收到消息以后会对其解码，还原成调用信息，然后在本地进行方法调用，然后把调用结果发送回调用端，这样一次 RPC 调用过程就完成了。在这个过程中，我们要考虑到一些问题：</p>
<ul>
<li>设计成什么样的调用模型？</li>
<li>调用信息通过什么样的方式序列化？通过哪种协议传输？性能如何？可靠性如何？</li>
<li>分布式系统中最关注的问题：出现 failure 如何应对？如何容错？</li>
<li>更为全面的服务治理功能（如服务发现、负载均衡等）</li>
</ul>
<p>我们一点一点来思考。第一点是设计成什么样的调用模型。常见的几种模型：</p>
<ul>
<li>服务代理。即实现一个服务接口，被调用端实现此服务接口，实现对应的方法逻辑，并写好 RPC 调用信息接收部分；调用端通过 RPC 模块获取一个服务代理实例，这个服务代理实例继承了服务接口并封装了相应的远程调用逻辑（包括消息的编码、解码、传输等）。调用端通过这个服务代理实例进行 RPC 调用。像 Vert.x Service Proxy 和 gRPC 都是这种模型。这样的 RPC 模块需要具备生成服务代理类的功能</li>
<li>直接调用，即设计特定的 API 用于 RPC 调用。比如 Golang 的 rpc 包，里面的 <code>Client</code> 就提供了一个 <code>Call</code> 方法用于任意RPC调用，调用者需要传入方法名称、参数以及返回值指针（异步模式下传入 callback handler）</li>
</ul>
<p>我更倾向于选择服务代理这种模型，因为服务代理这种模型在进行 RPC 调用的时候非常方便，但是需要 RPC 模块生成服务代理类（静态或动态生成），实现起来可能会麻烦些；当然 Go 的 rpc 包封装的也比较好，调用也比较方便，考虑到 Go 的类型系统，这已经不错了。。。</p>
<p>RPC 调用耗时会包含通信耗时和本地调用耗时。当网络状况不好的时候，RPC 调用可能会很长时间才能得到结果。对传统的同步 RPC 模式来说，这期间会阻塞调用者的调用线程。当需要进行大量 RPC 调用的时候，这种阻塞就伤不起了。这时候，异步 RPC 模式就派上用场了。我们可以对传统 RPC 模式稍加改造，把服务接口设计成异步模式的，即每个方法都要绑定一个回调函数，或利用 Future-Promise 模型返回一个 <code>Future</code>，或者直接上 reactive 模式（未来的趋势）。设计成异步、响应式模式以后，整个架构的灵活性就能得到很大的提升。</p>
<p>第二点是调用信息的序列化/反序列化以及传输。序列化主要分为文本（如 JSON, XML 等）和二进制（如 Thrift, Protocol 等）两种，不同的序列化策略性能不同，因此我们应该尽量选择性能高，同时便于开发的序列化策略。在大型项目中我们常用 Protobuf，性能比较好，支持多语言，但是需要单独定义 <code>.proto</code> 文件（可以利用 protostuff）；有的时候我们会选择 JSON，尽管效率不是很高但是方便，比如 Vert.x Service Proxy 就选择了 JSON 格式(底层依赖 Event Bus)。另一点就是传输协议的选择。通常情况下我们会选择 TCP 协议（以及各种基于 TCP 的应用层协议，如 HTTP/2）进行通信，当然用基于 AMQP 协议的消息队列也可以，两者都比较可靠。</p>
<p>这里还需提一点：如何高效地并发处理 request / response，这依赖于通信模块的实现。拿 Java 来说，基于 Netty NIO 或者 Java NIO / AIO 的 I/O 多路复用都可以很好地并发处理请求；而像 Go RPC 则是来一个 request 就创建一个 Goroutine 并在其中处理请求(Goroutine 作为轻量级用户态线程，创建性能消耗小)。</p>
<p>最后一点也是最重要的一点：实现容错，这也是分布式系统设计要考虑的一个核心。想象一下一次 RPC 调用过程中可能产生的各种 failure：</p>
<ul>
<li>网络拥塞</li>
<li>丢包，通信异常</li>
<li>服务提供端挂了，调用端得不到 response</li>
</ul>
<p>一种简单的应对方式是不断地超时重传，即 <em>at least once</em> 模式。调用端设置一个超时定时器，若一定时间内没有收到response就继续发送调用请求，直到收到response或请求次数达到阈值。这种模式会发送重复请求，因此只适用于幂等性的操作，即执行多次结果相同的操作，比如读取操作。当然服务提供端也可以实现对应的逻辑来检查重复的请求。</p>
<p>更符合我们期望的容错方案是 <em>at most once</em> 模式。<em>at most once</em> 模式要求服务提供端检查重复请求，如果检查到当前请求是重复请求则返回之前的调用结果。服务提供端需要缓存之前的调用结果。这里面有几点需要考虑：</p>
<ul>
<li>如何实现重传和重复请求检测？是依靠协议（如TCP的超时重传）还是自己实现？</li>
</ul>
<p>如果自己实现的话：</p>
<ul>
<li>如何检查重复请求？我们可以给每个请求生成一个独一无二的标识符(xid)，并且在重传请求的时候使用相同的xid进行重传。用伪代码可以表示为：</li>
</ul>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">if</span> (seen(xid)) &#123;</div><div class="line">  result = oldResult;</div><div class="line">&#125; <span class="keyword">else</span> &#123;</div><div class="line">  result = call(...);</div><div class="line">  oldResult = result;</div><div class="line">  setCurrentId(xid);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<ul>
<li>如何保证xid是独一无二的？可以考虑使用UUID或者不同seed下的随机数。</li>
<li>服务请求端需要在一个合适的时间丢弃掉保存的之前缓存的调用结果。</li>
<li>当某个RPC调用过程还正在执行时，如何应对另外的重复请求？这种情况可以设置一个flag用于标识是否正在执行。</li>
<li>如果服务调用端挂了并且重启怎么办？如果服务调用端将xid和调用结果缓存在内存中，那么保存的信息就丢失了。因此我们可以考虑将缓存信息定时写入硬盘，或者写入 replication server 中，当然这些情况就比较复杂了，涉及到高可用和一致性的问题。</li>
</ul>
<p>由此可见，虽然 RPC 模块看似比较简单，但是设计的时候要考虑的问题还是非常多的。尤其是在保证性能的基础上又要保证可靠性，还要保证开发者的易用性，这就需要细致地思考了。</p>
<h1 id="常见RPC模块实现">常见RPC模块实现</h1><p>这里我来简单总结一下用过的常见的几个 RPC 模块的使用及实现思路。</p>
<h2 id="Go_RPC">Go RPC</h2><p>Golang 的 <code>rpc</code> 包使用了 Go 自己的 gob 协议作为序列化协议(通过<code>encoding/gob</code>模块内的<code>Encoder</code>/<code>Decoder</code>进行编码和解码)，而传输协议可以直接使用TCP(<code>Dial</code>方法)或者使用HTTP(<code>DialHTTP</code>)方法。开发者需要在服务端定义struct并且实现各种方法，然后将struct注册到服务端。需要进行RPC调用的时候，我们就可以在调用端通过<code>Call</code>方法（同步）或者<code>Go</code>方法（异步）进行调用。同步模式下调用结果即为<code>reply</code>指针所指的对象，而异步模式则会在调用结果准备就绪后通知绑定的channel并执行处理。</p>
<p>在rpc包的实现中(<code>net/rpc/server.go</code>)，每个注册的服务类都被封装成了一个<code>service</code>结构体，而其中的每个方法则被封装成了一个<code>methodType</code>结构体：</p>
<figure class="highlight go"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">type</span> methodType <span class="keyword">struct</span> &#123;</div><div class="line">	sync.Mutex <span class="comment">// protects counters</span></div><div class="line">	method     reflect.Method</div><div class="line">	ArgType    reflect.Type</div><div class="line">	ReplyType  reflect.Type</div><div class="line">	numCalls   <span class="keyword">uint</span></div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">type</span> service <span class="keyword">struct</span> &#123;</div><div class="line">	name   <span class="keyword">string</span>                 <span class="comment">// name of service</span></div><div class="line">	rcvr   reflect.Value          <span class="comment">// receiver of methods for the service</span></div><div class="line">	typ    reflect.Type           <span class="comment">// type of the receiver</span></div><div class="line">	method <span class="keyword">map</span>[<span class="keyword">string</span>]*methodType <span class="comment">// registered methods</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>每个服务端都被封装成了一个<code>Server</code>结构体，其中的<code>serviceMap</code>存储着各个服务类的元数据：</p>
<figure class="highlight go"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">type</span> Server <span class="keyword">struct</span> &#123;</div><div class="line">	mu         sync.RWMutex <span class="comment">// protects the serviceMap</span></div><div class="line">	serviceMap <span class="keyword">map</span>[<span class="keyword">string</span>]*service</div><div class="line">	reqLock    sync.Mutex <span class="comment">// protects freeReq</span></div><div class="line">	freeReq    *Request</div><div class="line">	respLock   sync.Mutex <span class="comment">// protects freeResp</span></div><div class="line">	freeResp   *Response</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>RPC Server 处理调用请求的默认路径是 <code>/_goRPC_</code>。当请求到达时，Go就会调用<code>Server</code>结构体实现的<code>ServeHTTP</code>方法，经<code>ServeConn</code>方法传入gob codec预处理以后最终在<code>ServeCodec</code>方法内处理请求并进行调用：</p>
<figure class="highlight go"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">func</span> <span class="params">(server *Server)</span> <span class="title">ServeCodec</span><span class="params">(codec ServerCodec)</span></span> &#123;</div><div class="line">	sending := <span class="built_in">new</span>(sync.Mutex)</div><div class="line">	<span class="keyword">for</span> &#123;</div><div class="line">		service, mtype, req, argv, replyv, keepReading, err := server.readRequest(codec)</div><div class="line">		<span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</div><div class="line">			<span class="keyword">if</span> debugLog &amp;&amp; err != io.EOF &#123;</div><div class="line">				log.Println(<span class="string">"rpc:"</span>, err)</div><div class="line">			&#125;</div><div class="line">			<span class="keyword">if</span> !keepReading &#123;</div><div class="line">				<span class="keyword">break</span></div><div class="line">			&#125;</div><div class="line">			<span class="comment">// send a response if we actually managed to read a header.</span></div><div class="line">			<span class="keyword">if</span> req != <span class="literal">nil</span> &#123;</div><div class="line">				server.sendResponse(sending, req, invalidRequest, codec, err.Error())</div><div class="line">				server.freeRequest(req)</div><div class="line">			&#125;</div><div class="line">			<span class="keyword">continue</span></div><div class="line">		&#125;</div><div class="line">		<span class="keyword">go</span> service.call(server, sending, mtype, req, argv, replyv, codec)</div><div class="line">	&#125;</div><div class="line">	codec.Close()</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>如果成功读取请求数据，那么接下来 RPC Server 就会新建一个 Goroutine 用来在本地执行方法，并向调用端返回 response：</p>
<figure class="highlight go"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">func</span> <span class="params">(s *service)</span> <span class="title">call</span><span class="params">(server *Server, sending *sync.Mutex, mtype *methodType, req *Request, argv, replyv reflect.Value, codec ServerCodec)</span></span> &#123;</div><div class="line">	mtype.Lock()</div><div class="line">	mtype.numCalls++</div><div class="line">	mtype.Unlock()</div><div class="line">	function := mtype.method.Func</div><div class="line">	<span class="comment">// Invoke the method, providing a new value for the reply.</span></div><div class="line">	returnValues := function.Call([]reflect.Value&#123;s.rcvr, argv, replyv&#125;)</div><div class="line">	<span class="comment">// The return value for the method is an error.</span></div><div class="line">	errInter := returnValues[<span class="number">0</span>].Interface()</div><div class="line">	errmsg := <span class="string">""</span></div><div class="line">	<span class="keyword">if</span> errInter != <span class="literal">nil</span> &#123;</div><div class="line">		errmsg = errInter.(error).Error()</div><div class="line">	&#125;</div><div class="line">	server.sendResponse(sending, req, replyv.Interface(), codec, errmsg)</div><div class="line">	server.freeRequest(req)</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在执行调用的过程中应该注意并发问题，防止资源争用，修改数据时需要对数据加锁；至于方法的执行就是利用了Go的反射机制。调用完以后，RPC Server 接着调用<code>sendResponse</code>方法发送response，其中写入response的时候同样需要加锁，防止资源争用。</p>
<h2 id="gRPC">gRPC</h2><p>gRPC 是 Google 开源的一个通用的 RPC 框架，支持 C, Java 和 Go 等语言。既然是 Google 出品，序列化协议必然用 protobuf 啦（毕竟高效），传输协议使用 HTTP/2，更为通用，性能也还可以。开发时需要在 <code>.proto</code> 文件里定义数据类型以及服务接口，然后配上 protoc 的 gRPC 插件就能够自动生成各个语言的服务接口和代理类。</p>
<h2 id="Vert-x_Service_Proxy">Vert.x Service Proxy</h2><p>Vert.x Service Proxy 是 Vert.x 的一个异步 RPC 组件，支持通过各种 JVM 语言(Java, Scala, JS, JRuby, Groovy等)进行 RPC 调用。使用 Vert.x Service Proxy 时我们只需要按照异步开发模式编写服务接口，加上相应的注解，Vert.x Service Proxy 就会自动生成相应的服务代理类和服务调用处理类。Vert.x Service Proxy 底层借助 Event Bus 进行通信，调用时将调用消息包装成 JSON 数据然后通过 Event Bus 传输到服务端，得到结果后再返回给调用端。Vert.x 的一大特性就是异步、响应式编程，因此 Vert.x Service Proxy 的 RPC 模型为异步 RPC，用起来非常方便。几个异步过程可以通过各种组合子串成一串，妥妥的 reactive programming 的风格~</p>
<p>更多的关于 Vert.x Service Proxy 的实现原理的内容可以看这一篇：<a href="http://www.sczyh30.com/posts/Vert-x/vertx-advanced-async-rpc/">Vert.x 技术内幕 | 异步 RPC 实现原理</a>。</p>
<p>PS: 我经常吐槽 Vert.x Service Proxy 这个名字，因为光看名字很多人不知道它可以用来实现 RPC，导致了很多人以为 Vert.x 不能做 RPC，应该改名叫 Vert.x Async RPC 比较合适。。。当然它还有很大的改进空间，主要是被 Vert.x Event Bus 的性能和可靠性给拖累了。。。</p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;RPC 是分布式系统中不可缺少的一部分。之前接触过几种 RPC 组件，这里就总结一下常见 RPC 模块的设计思想和实现。最后我们来设计一个可以方便进行 RPC 调用的 RPC 模块。&lt;/p&gt;
&lt;h1 id=&quot;RPC_模块设计需要考虑的问题&quot;&gt;RPC 模块设计需要考虑的问题&lt;
    
    </summary>
    
      <category term="Distributed System" scheme="http://www.sczyh30.com/categories/Distributed-System/"/>
    
    
      <category term="分布式系统" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="架构" scheme="http://www.sczyh30.com/tags/%E6%9E%B6%E6%9E%84/"/>
    
      <category term="RPC" scheme="http://www.sczyh30.com/tags/RPC/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x 技术内幕 | Event Bus 源码分析 (集群模式)</title>
    <link href="http://www.sczyh30.com/posts/Vert-x/vertx-advanced-clustered-event-bus-internal/"/>
    <id>http://www.sczyh30.com/posts/Vert-x/vertx-advanced-clustered-event-bus-internal/</id>
    <published>2016-09-04T16:00:00.000Z</published>
    <updated>2016-11-24T06:07:28.000Z</updated>
    
    <content type="html"><![CDATA[<p><a href="http://www.sczyh30.com/posts/Vert-x/vertx-advanced-local-event-bus-internal/">上篇文章</a>中我们探索了Local模式下Event Bus的源码，在这篇文章中我们来探索一下Vert.x中的Clustered Event Bus是如何实现的。对应的Vert.x版本为<strong>3.3.2</strong>。</p>
<h1 id="集群模式介绍">集群模式介绍</h1><p>我们先来简单地介绍一下集群模式下Event Bus的基本原理。</p>
<p>我们可以通过集群模式下的Event Bus在不同的服务器之间进行通信，其本质为TCP通信。Vert.x集群模式需要一个集群管理器（默认为<code>HazelcastClusterManager</code>）来管理集群的状态，存储元数据。当我们在某个节点A给集群模式的Event Bus绑定一个对应地址<code>address</code>的<code>consumer</code>的时候，Event Bus会将此节点的<code>ServerID</code>（包含<code>host</code>和<code>port</code>信息）存储至集群管理器的共享Map中，<code>key</code>为绑定的地址<code>address</code>，value为绑定了此地址<code>address</code>的所有结点的<code>ServerID</code>集合（可以看作是具有负载均衡功能的<code>Set</code>）。集群中的所有节点都可以从集群管理器中获取Map记录。并且绑定consumer的同时节点A会建立一个<code>NetServer</code>接收数据。这样，我们再通过另一个结点B向此地址<code>address</code>发送消息的时候，B就会从集群管理器中取出此地址对应的<code>ServerID</code>集合，并根据是点对点发送还是发布，根据相应的策略创建<code>NetClient</code>执行消息分发逻辑。这样，对应的<code>NetServer</code>收到数据后会对其进行解码然后在本地进行消息的处理。</p>
<p>集群模式下我们还需要注意几个问题：</p>
<ul>
<li>某个节点挂了怎么办？</li>
<li>如何确保结点的高可用性？</li>
</ul>
<p>当某个节点挂掉的时候，其连接将会不可用，集群管理器就会将此节点的信息从集群中移除，并且传播到所有的节点删除对应缓存的信息，这样发消息的时候就不会发送到挂掉的无效节点处。至于高可用性，Vert.x提供了高可用管理器<code>HAManager</code>用于实现高可用性，在发生故障时能够快速failover，详情可见<a href="http://vertx.io/docs/vertx-core/java/#_high_availability_and_fail_over" target="_blank" rel="external">官方文档</a>。</p>
<p>好了，下面我们就来分析一下Clustered Event Bus的源码。集群模式下Event Bus的类型为<code>ClusteredEventBus</code>，它继承了单机模式的<code>EventBusImpl</code>类。其初始化过程与Local模式大同小异，因此这里就直接分析发送和接受消息相关的逻辑了。</p>
<h1 id="绑定MessageConsumer">绑定MessageConsumer</h1><p>我们还是先来看<code>consumer</code>方法的逻辑。前面的调用逻辑都和Local模式下相同，可以参考之前的文章。不同之处在添加记录的地方。Cluster模式下Event Bus需要将当前机器的位置存储至Map中并且传播至集群内的所有节点，因此<code>ClusteredEventBus</code>重写了四个参数版本的<code>addRegistration</code>方法（之前在<code>EventBusImpl</code>类中这个版本的方法用处不大，这里用处就大了）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">addRegistration</span><span class="params">(<span class="keyword">boolean</span> newAddress, String address,</span></span></div><div class="line">                                   <span class="keyword">boolean</span> replyHandler, <span class="keyword">boolean</span> localOnly,</div><div class="line">                                   Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler) &#123;</div><div class="line">  <span class="keyword">if</span> (newAddress &amp;&amp; subs != <span class="keyword">null</span> &amp;&amp; !replyHandler &amp;&amp; !localOnly) &#123;</div><div class="line">    <span class="comment">// Propagate the information</span></div><div class="line">    subs.add(address, serverID, completionHandler);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    completionHandler.handle(Future.succeededFuture());</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>如果要绑定<code>MessageConsumer</code>对应的地址在本地中没有注册过，并且不是Event Bus自动生成的reply consumer，并且允许在集群范围内传播的话，Event Bus就会将当前机器的位置添加到集群内的记录<code>subs</code>中。<code>subs</code>的类型为<code>AsyncMultiMap</code>:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> AsyncMultiMap&lt;String, ServerID&gt; subs;</div></pre></td></tr></table></figure>
<p>ClusteredEventBus启动时会对其进行初始化：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">clusterManager.&lt;String, ServerID&gt;getAsyncMultiMap(SUBS_MAP_NAME, ar2 -&gt; &#123;</div><div class="line">  <span class="keyword">if</span> (ar2.succeeded()) &#123;</div><div class="line">    subs = ar2.result();</div><div class="line">    <span class="comment">// 其他代码暂略。。。</span></div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    <span class="comment">// 代码略。。。</span></div><div class="line">  &#125;</div><div class="line">&#125;);</div></pre></td></tr></table></figure>
<p>从名字就可以看出来，<code>AsyncMultiMap</code>允许<strong>一键多值</strong>，并且其变动可以<strong>在集群范围内传播</strong>。由于<code>AsyncMultiMap</code>是<strong>集群范围内</strong>的，因此对其操作都是异步的。在这里我们可以简单地把它看作是一个<code>Map&lt;String, ChoosableIterable&lt;ServerID&gt;&gt;</code>类型的键值对，其中<code>ChoosableIterable</code>与之前见到过的<code>Handlers</code>类似，属于可以通过轮询算法获取某一元素的集合。<code>subs</code>的key为绑定的地址，value为绑定此地址的机器位置的集合。机器的位置用<code>ServerID</code>表示，里面包含了该机器的<code>host</code>和<code>port</code>。这样，每当我们向某个地址绑定一个<code>MessageConsumer</code>的时候，绑定consumer的<code>ServerID</code>就会被记录到集群中并与地址相对应，其它机器在向此地址发送（或发布）消息的时候，Event Bus就可以从集群中获取在此地址上绑定了consumer的所有<code>ServerID</code>，再根据相应的策略选出合适的<code>ServerID</code>建立TCP通信将数据发送至对应机器中，对应机器收到消息后解码并在本地对其进行处理。</p>
<p>这里面还需要注意一点：我们可以在<code>EventBusOptions</code>中指定<code>ServerID</code>的<code>port</code>和<code>host</code>，若不指定则<code>port</code>将随机分配(<code>NetServer</code>的特性)。</p>
<p>剩下的过程也就大同小异了。至于<code>unregister</code>方法，无非就是将底层的<code>removeRegistration</code>方法重写，从<code>subs</code>中删除对应的<code>ServerID</code>并传播至其它节点：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">removeRegistration</span><span class="params">(HandlerHolder lastHolder, String address,</span></span></div><div class="line">                                      Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler) &#123;</div><div class="line">  <span class="keyword">if</span> (lastHolder != <span class="keyword">null</span> &amp;&amp; subs != <span class="keyword">null</span> &amp;&amp; !lastHolder.isLocalOnly()) &#123;</div><div class="line">    removeSub(address, serverID, completionHandler);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    callCompletionHandlerAsync(completionHandler);</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">removeSub</span><span class="params">(String subName, ServerID theServerID, Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler)</span> </span>&#123;</div><div class="line">  subs.remove(subName, theServerID, ar -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (!ar.succeeded()) &#123;</div><div class="line">      log.error(<span class="string">"Failed to remove sub"</span>, ar.cause());</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">if</span> (ar.result()) &#123;</div><div class="line">        <span class="keyword">if</span> (completionHandler != <span class="keyword">null</span>) &#123;</div><div class="line">          completionHandler.handle(Future.succeededFuture());</div><div class="line">        &#125;</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="keyword">if</span> (completionHandler != <span class="keyword">null</span>) &#123;</div><div class="line">          completionHandler.handle(Future.failedFuture(<span class="string">"sub not found"</span>));</div><div class="line">        &#125;</div><div class="line">      &#125;</div><div class="line"></div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<h1 id="消息的发送/发布">消息的发送/发布</h1><p>集群模式下的消息与本地模式下的消息不同。集群模式下的消息实体类型为<code>ClusteredMessage</code>，它继承了<code>MessageImpl</code>消息实体类，并且根据远程传输的特性实现了一种Wire Protocol用于远程传输消息，并负责消息的编码和解码。具体的实现就不展开说了，如果有兴趣的话可以阅读<code>ClusteredMessage</code>类中相关方法的实现。</p>
<p>我们上篇文章提到过，Event Bus底层通过<code>createMessage</code>方法创建消息。因此<code>ClusteredEventBus</code>里就对此方法进行了重写，当然改动就是把<code>MessageImpl</code>替换成了<code>ClusteredMessage</code>:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">protected</span> MessageImpl <span class="title">createMessage</span><span class="params">(<span class="keyword">boolean</span> send, String address, MultiMap headers, Object body, String codecName)</span> </span>&#123;</div><div class="line">  Objects.requireNonNull(address, <span class="string">"no null address accepted"</span>);</div><div class="line">  MessageCodec codec = codecManager.lookupCodec(body, codecName);</div><div class="line">  <span class="meta">@SuppressWarnings</span>(<span class="string">"unchecked"</span>)</div><div class="line">  ClusteredMessage msg = <span class="keyword">new</span> ClusteredMessage(serverID, address, <span class="keyword">null</span>, headers, body, codec, send, <span class="keyword">this</span>);</div><div class="line">  <span class="keyword">return</span> msg;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>接下来就是消息的发送逻辑了。<code>ClusteredEventBus</code>重写了<code>sendOrPub</code>方法，此方法存在于<code>SendContextImpl</code>类中的<code>next</code>方法:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">next</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (iter.hasNext()) &#123;</div><div class="line">    Handler&lt;SendContext&gt; handler = iter.next();</div><div class="line">    <span class="keyword">try</span> &#123;</div><div class="line">      handler.handle(<span class="keyword">this</span>);</div><div class="line">    &#125; <span class="keyword">catch</span> (Throwable t) &#123;</div><div class="line">      log.error(<span class="string">"Failure in interceptor"</span>, t);</div><div class="line">    &#125;</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    sendOrPub(<span class="keyword">this</span>);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们来看一下<code>ClusteredEventBus</code>是如何进行集群内消息的分发的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">sendOrPub</span><span class="params">(SendContextImpl&lt;T&gt; sendContext)</span> </span>&#123;</div><div class="line">  String address = sendContext.message.address();</div><div class="line">  Handler&lt;AsyncResult&lt;ChoosableIterable&lt;ServerID&gt;&gt;&gt; resultHandler = asyncResult -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (asyncResult.succeeded()) &#123;</div><div class="line">      ChoosableIterable&lt;ServerID&gt; serverIDs = asyncResult.result();</div><div class="line">      <span class="keyword">if</span> (serverIDs != <span class="keyword">null</span> &amp;&amp; !serverIDs.isEmpty()) &#123;</div><div class="line">        sendToSubs(serverIDs, sendContext);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        metrics.messageSent(address, !sendContext.message.send(), <span class="keyword">true</span>, <span class="keyword">false</span>);</div><div class="line">        deliverMessageLocally(sendContext);</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      log.error(<span class="string">"Failed to send message"</span>, asyncResult.cause());</div><div class="line">    &#125;</div><div class="line">  &#125;;</div><div class="line">  <span class="keyword">if</span> (Vertx.currentContext() == <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="comment">// Guarantees the order when there is no current context</span></div><div class="line">    sendNoContext.runOnContext(v -&gt; &#123;</div><div class="line">      subs.get(address, resultHandler);</div><div class="line">    &#125;);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    subs.get(address, resultHandler);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先Event Bus需要从传入的<code>sendContext</code>中获取要发送至的地址。接着Event Bus需要从集群管理器中获取在此地址上绑定consumer的所有<code>ServerID</code>，这个过程是异步的，并且需要在Vert.x Context中执行。如果获取记录成功，我们会得到一个可通过轮询算法获取<code>ServerID</code>的集合(类型为<code>ChoosableIterable&lt;ServerID&gt;</code>)。如果集合为空，则代表集群内其它节点没有在此地址绑定consumer（或者由于一致性问题没有同步），Event Bus就将消息通过<code>deliverMessageLocally</code>方法在本地进行相应的分发。<code>deliverMessageLocally</code>方法的逻辑之前我们已经详细讲过了，这里就不再细说了；如果集合不为空，Event Bus就调用<code>sendToSubs</code>方法进行下一步操作：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">sendToSubs</span><span class="params">(ChoosableIterable&lt;ServerID&gt; subs, SendContextImpl&lt;T&gt; sendContext)</span> </span>&#123;</div><div class="line">  String address = sendContext.message.address();</div><div class="line">  <span class="keyword">if</span> (sendContext.message.send()) &#123;</div><div class="line">    <span class="comment">// Choose one</span></div><div class="line">    ServerID sid = subs.choose();</div><div class="line">    <span class="keyword">if</span> (!sid.equals(serverID)) &#123;  <span class="comment">//We don't send to this node</span></div><div class="line">      metrics.messageSent(address, <span class="keyword">false</span>, <span class="keyword">false</span>, <span class="keyword">true</span>);</div><div class="line">      sendRemote(sid, sendContext.message);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      metrics.messageSent(address, <span class="keyword">false</span>, <span class="keyword">true</span>, <span class="keyword">false</span>);</div><div class="line">      deliverMessageLocally(sendContext);</div><div class="line">    &#125;</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    <span class="comment">// Publish</span></div><div class="line">    <span class="keyword">boolean</span> local = <span class="keyword">false</span>;</div><div class="line">    <span class="keyword">boolean</span> remote = <span class="keyword">false</span>;</div><div class="line">    <span class="keyword">for</span> (ServerID sid : subs) &#123;</div><div class="line">      <span class="keyword">if</span> (!sid.equals(serverID)) &#123;  <span class="comment">//We don't send to this node</span></div><div class="line">        remote = <span class="keyword">true</span>;</div><div class="line">        sendRemote(sid, sendContext.message);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        local = <span class="keyword">true</span>;</div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">    metrics.messageSent(address, <span class="keyword">true</span>, local, remote);</div><div class="line">    <span class="keyword">if</span> (local) &#123;</div><div class="line">      deliverMessageLocally(sendContext);</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里就到了分<code>send</code>和<code>publish</code>的时候了。如果发送消息的模式为点对点模式(<code>send</code>)，Event Bus会从给的的集合中通过轮询算法获取一个<code>ServerID</code>。然后Event Bus会检查获取到的<code>ServerID</code>是否与本机<code>ServerID</code>相同，如果相同则代表在一个机子上，直接记录metrics信息并且调用<code>deliverMessageLocally</code>方法往本地发送消息即可；如果不相同，Event Bus就会调用<code>sendRemote</code>方法执行真正的远程消息发送逻辑。发布订阅模式的逻辑与其大同小异，只不过需要遍历一下<code>ChoosableIterable&lt;ServerID&gt;</code>集合，然后依次执行之前讲过的逻辑。注意如果要在本地发布消息只需要发一次。</p>
<p>真正的远程消息发送逻辑在<code>sendRemote</code>方法中:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">sendRemote</span><span class="params">(ServerID theServerID, MessageImpl message)</span> </span>&#123;</div><div class="line">  ConnectionHolder holder = connections.get(theServerID);</div><div class="line">  <span class="keyword">if</span> (holder == <span class="keyword">null</span>) &#123;</div><div class="line">    holder = <span class="keyword">new</span> ConnectionHolder(<span class="keyword">this</span>, theServerID, options);</div><div class="line">    ConnectionHolder prevHolder = connections.putIfAbsent(theServerID, holder);</div><div class="line">    <span class="keyword">if</span> (prevHolder != <span class="keyword">null</span>) &#123;</div><div class="line">      <span class="comment">// Another one sneaked in</span></div><div class="line">      holder = prevHolder;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      holder.connect();</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">  holder.writeMessage((ClusteredMessage)message);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>一开始我们就提到过，节点之间通过Event Bus进行通信的本质是TCP，因此这里需要创建一个<code>NetClient</code>作为TCP服务端，连接到之前获取的<code>ServerID</code>对应的节点然后将消息通过TCP协议发送至接收端。这里Vert.x用一个封装类<code>ConnectionHolder</code>对<code>NetClient</code>进行了一些封装。</p>
<p><code>ClusteredEventBus</code>中维持着一个<code>connections</code>哈希表对用于保存<code>ServerID</code>对应的连接<code>ConnectionHolder</code>。在<code>sendRemote</code>方法中,Event Bus首先会从<code>connections</code>中获取<code>ServerID</code>对应的连接。如果获取不到就创建连接并将其添加至<code>connections</code>记录中并调用对应<code>ConnectionHolder</code>的<code>connect</code>方法建立连接；最后调用<code>writeMessage</code>方法将消息编码后通过TCP发送至对应的接收端。</p>
<p>那么<code>ConnectionHolder</code>是如何实现的呢？我们来看一下其构造函数：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div></pre></td><td class="code"><pre><div class="line">ConnectionHolder(ClusteredEventBus eventBus, ServerID serverID, EventBusOptions options) &#123;</div><div class="line">  <span class="keyword">this</span>.eventBus = eventBus;</div><div class="line">  <span class="keyword">this</span>.serverID = serverID;</div><div class="line">  <span class="keyword">this</span>.vertx = eventBus.vertx();</div><div class="line">  <span class="keyword">this</span>.metrics = eventBus.getMetrics();</div><div class="line">  NetClientOptions clientOptions = <span class="keyword">new</span> NetClientOptions(options.toJson());</div><div class="line">  ClusteredEventBus.setCertOptions(clientOptions, options.getKeyCertOptions());</div><div class="line">  ClusteredEventBus.setTrustOptions(clientOptions, options.getTrustOptions());</div><div class="line">  client = <span class="keyword">new</span> NetClientImpl(eventBus.vertx(), clientOptions, <span class="keyword">false</span>);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到<code>ConnectionHolder</code>初始化的时候会创建一个<code>NetClient</code>作为TCP请求端，而请求的对象就是接收端的<code>NetServer</code>(后边会讲)，客户端配置已经在<code>EventBusOptions</code>中事先配置好了。我们来看看<code>connect</code>方法是如何建立连接的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">connect</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (connected) &#123;</div><div class="line">    <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"Already connected"</span>);</div><div class="line">  &#125;</div><div class="line">  client.connect(serverID.port, serverID.host, res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.succeeded()) &#123;</div><div class="line">      connected(res.result());</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      close();</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到这里很简单地调用了<code>NetClient#connect</code>方法建立TCP连接，如果建立连接成功的话会得到一个<code>NetSocket</code>对象。Event Bus接着将其传至<code>connected</code>方法中进行处理：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">connected</span><span class="params">(NetSocket socket)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.socket = socket;</div><div class="line">  connected = <span class="keyword">true</span>;</div><div class="line">  socket.exceptionHandler(t -&gt; close());</div><div class="line">  socket.closeHandler(v -&gt; close());</div><div class="line">  socket.handler(data -&gt; &#123;</div><div class="line">    <span class="comment">// Got a pong back</span></div><div class="line">    vertx.cancelTimer(timeoutID);</div><div class="line">    schedulePing();</div><div class="line">  &#125;);</div><div class="line">  <span class="comment">// Start a pinger</span></div><div class="line">  schedulePing();</div><div class="line">  <span class="keyword">for</span> (ClusteredMessage message : pending) &#123;</div><div class="line">    Buffer data = message.encodeToWire();</div><div class="line">    metrics.messageWritten(message.address(), data.length());</div><div class="line">    socket.write(data);</div><div class="line">  &#125;</div><div class="line">  pending.clear();</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先Event Bus通过<code>exceptionHandler</code>和<code>closeHandler</code>方法给连接对应的<code>NetSocket</code>绑定异常回调和连接关闭回调，触发的时候都调用<code>close</code>方法关闭连接；为了保证不丢失连接，消息发送方每隔一段时间就需要对消息接收方发送一次心跳包（<code>PING</code>），如果消息接收方在一定时间内没有回复，那么就认为连接丢失，调用<code>close</code>方法关闭连接。心跳检测的逻辑在<code>schedulePing</code>方法中，比较清晰，这里就不详细说了。大家会发现<code>ConnectionHolder</code>里也有个消息队列（缓冲区）<code>pending</code>，并且这里会将队列中的消息依次通过TCP发送至接收端。为什么需要这样呢？其实，这要从创建TCP客户端说起。创建TCP客户端这个过程应该是异步的，需要消耗一定时间，而<code>ConnectionHolder</code>中封装的<code>connect</code>方法却是同步式的。前面我们刚刚看过，通过<code>connect</code>方法建立连接以后会接着调用<code>writeMessage</code>方法发送消息，而这时候客户端连接可能还没建立，因此需要这么个缓冲区先存着，等着连接建立了再一块发送出去（存疑：为什么不将<code>connect</code>方法直接设计成异步的？）。</p>
<p>至于发送消息的<code>writeMessage</code>方法，其逻辑一目了然：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">writeMessage</span><span class="params">(ClusteredMessage message)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (connected) &#123;</div><div class="line">    Buffer data = message.encodeToWire();</div><div class="line">    metrics.messageWritten(message.address(), data.length());</div><div class="line">    socket.write(data);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    <span class="keyword">if</span> (pending == <span class="keyword">null</span>) &#123;</div><div class="line">      pending = <span class="keyword">new</span> ArrayDeque&lt;&gt;();</div><div class="line">    &#125;</div><div class="line">    pending.add(message);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>如果连接已建立，Event Bus就会调用对应<code>ClusteredMessage</code>的<code>encodeToWire</code>方法将其转化为字节流<code>Buffer</code>，然后记录metrics信息，最后通过<code>socket</code>的<code>write</code>方法将消息写入到Socket中，这样消息就从发送端通过TCP发送到了接收端。如果连接未建立，就如之前讲的那样，先把消息存到消息队列中，等连接建立了再一块发出去。</p>
<p>这样，Clustered Event Bus下消息的发送逻辑就理清楚了。下面我们看一下接收端是如何接收消息并在本地进行消息的处理的。</p>
<h1 id="消息的接收">消息的接收</h1><p>一开始我们提到过，每个节点的Clustered Event Bus在启动时都会创建一个<code>NetServer</code>作为接收消息的TCP服务端。TCP Server的<code>port</code>和<code>host</code>可以在<code>EventBusOptions</code>中指定，如果不指定的话默认随机分配<code>port</code>，然后Event Bus会根据<code>NetServer</code>的配置来生成当前节点的<code>ServerID</code>。</p>
<p>创建TCP Server的逻辑在<code>start</code>方法中，与接受消息有关的逻辑就是这一句：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">server.connectHandler(getServerHandler());</div></pre></td></tr></table></figure>
<p>我们知道，<code>NetServer</code>的<code>connectHandler</code>方法用于绑定对服务端Socket的处理函数，而这里绑定的处理函数是由<code>getServerHandler</code>方法生成的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> Handler&lt;NetSocket&gt; <span class="title">getServerHandler</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">return</span> socket -&gt; &#123;</div><div class="line">    RecordParser parser = RecordParser.newFixed(<span class="number">4</span>, <span class="keyword">null</span>);</div><div class="line">    Handler&lt;Buffer&gt; handler = <span class="keyword">new</span> Handler&lt;Buffer&gt;() &#123;</div><div class="line">      <span class="keyword">int</span> size = -<span class="number">1</span>;</div><div class="line">      <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">handle</span><span class="params">(Buffer buff)</span> </span>&#123;</div><div class="line">        <span class="keyword">if</span> (size == -<span class="number">1</span>) &#123;</div><div class="line">          size = buff.getInt(<span class="number">0</span>);</div><div class="line">          parser.fixedSizeMode(size);</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">          ClusteredMessage received = <span class="keyword">new</span> ClusteredMessage();</div><div class="line">          received.readFromWire(buff, codecManager);</div><div class="line">          metrics.messageRead(received.address(), buff.length());</div><div class="line">          parser.fixedSizeMode(<span class="number">4</span>);</div><div class="line">          size = -<span class="number">1</span>;</div><div class="line">          <span class="keyword">if</span> (received.codec() == CodecManager.PING_MESSAGE_CODEC) &#123;</div><div class="line">            <span class="comment">// Just send back pong directly on connection</span></div><div class="line">            socket.write(PONG);</div><div class="line">          &#125; <span class="keyword">else</span> &#123;</div><div class="line">            deliverMessageLocally(received);</div><div class="line">          &#125;</div><div class="line">        &#125;</div><div class="line">      &#125;</div><div class="line">    &#125;;</div><div class="line">    parser.setOutput(handler);</div><div class="line">    socket.handler(parser);</div><div class="line">  &#125;;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>逻辑非常清晰。这里Event Bus使用了<code>RecordParser</code>来获取发送过来的对应长度的<code>Buffer</code>，并将其绑定在<code>NetServer</code>的Socket上。真正的解析<code>Buffer</code>并处理的逻辑在内部的<code>handler</code>中。之前<code>ClusteredMessage</code>中的Wire Protocol规定<code>Buffer</code>的首部第一个<code>int</code>值为要发送<code>Buffer</code>的长度（逻辑见<code>ClusteredMessage#encodeToWire</code>方法），因此这里首先获取长度，然后给<code>parser</code>设定正确的fixed size，这样<code>parser</code>就可以截取正确长度的<code>Buffer</code>流了。下面Event Bus会创建一个空的<code>ClusteredMessage</code>，然后调用其<code>readFromWire</code>方法从<code>Buffer</code>中重建消息。当然这里还要记录消息已经读取的metrics信息。接着检测收到的消息实体类型是否为心跳检测包(<code>PING</code>)，如果是的话就发送回ACK消息(<code>PONG</code>)；如果不是心跳包，则代表是正常的消息，Event Bus就调用我们熟悉的<code>deliverMessageLocally</code>函数在本地进行分发处理，接下来的过程就和Local模式一样了。</p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;&lt;a href=&quot;http://www.sczyh30.com/posts/Vert-x/vertx-advanced-local-event-bus-internal/&quot;&gt;上篇文章&lt;/a&gt;中我们探索了Local模式下Event Bus的源码，在这篇文章中我们来探索一下Ve
    
    </summary>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/categories/Vert-x/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
      <category term="源码分析" scheme="http://www.sczyh30.com/tags/%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x 技术内幕 | Event Bus 源码分析 (Local模式)</title>
    <link href="http://www.sczyh30.com/posts/Vert-x/vertx-advanced-local-event-bus-internal/"/>
    <id>http://www.sczyh30.com/posts/Vert-x/vertx-advanced-local-event-bus-internal/</id>
    <published>2016-09-02T16:00:00.000Z</published>
    <updated>2018-09-23T14:15:44.000Z</updated>
    
    <content type="html"><![CDATA[<p>Event Bus是Vert.x的“神经系统”，是最为关键的几个部分之一。今天我们就来探索一下Event Bus的实现原理。本篇分析的是Local模式的Event Bus，对应的Vert.x版本为<strong>3.3.2</strong>。</p>
<p>本文假定读者有一定的并发编程基础以及Vert.x使用基础，并且对Vert.x的线程模型以及<a href="http://vertx.io/docs/vertx-core/java/#streams" target="_blank" rel="external">back-pressure</a>有所了解。</p>
<h1 id="Local_Event_Bus的创建">Local Event Bus的创建</h1><p>一般情况下，我们通过<code>Vertx</code>的<code>eventBus</code>方法来创建或获取一个<code>EventBus</code>实例：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">Vertx vertx = Vertx.vertx();</div><div class="line">EventBus eventBus = vertx.eventBus();</div></pre></td></tr></table></figure>
<p><code>eventBus</code>方法定义于<code>Vertx</code>接口中，我们来看一下其位于<code>VertxImpl</code>类中的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> EventBus <span class="title">eventBus</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (eventBus == <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="comment">// If reading from different thread possibility that it's been set but not visible - so provide</span></div><div class="line">    <span class="comment">// memory barrier</span></div><div class="line">    <span class="keyword">synchronized</span> (<span class="keyword">this</span>) &#123;</div><div class="line">      <span class="keyword">return</span> eventBus;</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">return</span> eventBus;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到此方法返回<code>VertxImpl</code>实例中的<code>eventBus</code>成员，同时需要注意并发可见性问题。那么<code>eventBus</code>成员是何时初始化的呢？答案在<code>VertxImpl</code>类的构造函数中。这里截取相关逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">if</span> (options.isClustered()) &#123;</div><div class="line">  <span class="comment">// 集群模式相关逻辑</span></div><div class="line">&#125; <span class="keyword">else</span> &#123;</div><div class="line">  <span class="keyword">this</span>.clusterManager = <span class="keyword">null</span>;</div><div class="line">  createAndStartEventBus(options, resultHandler);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到<code>VertxImpl</code>内部是通过<code>createAndStartEventBus</code>方法来初始化<code>eventBus</code>的。我们来看一下其逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">createAndStartEventBus</span><span class="params">(VertxOptions options, Handler&lt;AsyncResult&lt;Vertx&gt;&gt; resultHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (options.isClustered()) &#123;</div><div class="line">    eventBus = <span class="keyword">new</span> ClusteredEventBus(<span class="keyword">this</span>, options, clusterManager, haManager);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    eventBus = <span class="keyword">new</span> EventBusImpl(<span class="keyword">this</span>);</div><div class="line">  &#125;</div><div class="line">  eventBus.start(ar2 -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (ar2.succeeded()) &#123;</div><div class="line">      <span class="comment">// If the metric provider wants to use the event bus, it cannot use it in its constructor as the event bus</span></div><div class="line">      <span class="comment">// may not be initialized yet. We invokes the eventBusInitialized so it can starts using the event bus.</span></div><div class="line">      metrics.eventBusInitialized(eventBus);</div><div class="line"></div><div class="line">      <span class="keyword">if</span> (resultHandler != <span class="keyword">null</span>) &#123;</div><div class="line">        resultHandler.handle(Future.succeededFuture(<span class="keyword">this</span>));</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      log.error(<span class="string">"Failed to start event bus"</span>, ar2.cause());</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到此方法通过<code>eventBus = new EventBusImpl(this)</code>将<code>eventBus</code>进行了初始化(Local模式为<code>EventBusImpl</code>)，并且调用<code>eventBus</code>的<code>start</code>方法对其进行一些额外的初始化工作。我们来看一下<code>EventBusImpl</code>类的<code>start</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">start</span><span class="params">(Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (started) &#123;</div><div class="line">    <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"Already started"</span>);</div><div class="line">  &#125;</div><div class="line">  started = <span class="keyword">true</span>;</div><div class="line">  completionHandler.handle(Future.succeededFuture());</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先初始化过程需要防止race condition，因此方法为<code>synchronized</code>的。该方法仅仅将<code>EventBusImpl</code>类中的一个<code>started</code>标志位设为<code>true</code>来代表Event Bus已启动。注意<code>started</code>标志位为<code>volatile</code>的，这样可以保证其可见性，确保其它线程通过<code>checkStarted</code>方法读到的<code>started</code>结果总是最新的。设置完<code>started</code>标志位后，Vert.x会接着调用传入的<code>completionHandler</code>处理函数，也就是上面我们在<code>createAndStartEventBus</code>方法中看到的 —— 调用<code>metrics</code>成员的<code>eventBusInitialized</code>方法以便Metrics类可以在Event Bus初始化完毕后使用它（不过默认情况下此方法的逻辑为空）。</p>
<p>可以看到初始化过程还是比较简单的，我们接下来先来看看订阅消息 —— <code>consumer</code>方法的逻辑。</p>
<h1 id="consume">consume</h1><p>我们来看一下<code>consumer</code>方法的逻辑，其原型位于<code>EventBus</code>接口中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">&lt;T&gt; <span class="function">MessageConsumer&lt;T&gt; <span class="title">consumer</span><span class="params">(String address)</span></span>;</div><div class="line">&lt;T&gt; <span class="function">MessageConsumer&lt;T&gt; <span class="title">consumer</span><span class="params">(String address, Handler&lt;Message&lt;T&gt;&gt; handler)</span></span>;</div></pre></td></tr></table></figure>
<p>其实现位于<code>EventBusImpl</code>类中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> &lt;T&gt; <span class="function">MessageConsumer&lt;T&gt; <span class="title">consumer</span><span class="params">(String address, Handler&lt;Message&lt;T&gt;&gt; handler)</span> </span>&#123;</div><div class="line">  Objects.requireNonNull(handler, <span class="string">"handler"</span>);</div><div class="line">  MessageConsumer&lt;T&gt; consumer = consumer(address);</div><div class="line">  consumer.handler(handler);</div><div class="line">  <span class="keyword">return</span> consumer;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先要确保传入的<code>handler</code>不为空，然后Vert.x会调用只接受一个<code>address</code>参数的<code>consumer</code>方法获取对应的<code>MessageConsumer</code>，最后给获取到的<code>MessageConsumer</code>绑定上传入的<code>handler</code>。我们首先来看一下另一个<code>consumer</code>方法的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> &lt;T&gt; <span class="function">MessageConsumer&lt;T&gt; <span class="title">consumer</span><span class="params">(String address)</span> </span>&#123;</div><div class="line">  checkStarted();</div><div class="line">  Objects.requireNonNull(address, <span class="string">"address"</span>);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">new</span> HandlerRegistration&lt;&gt;(vertx, metrics, <span class="keyword">this</span>, address, <span class="keyword">null</span>, <span class="keyword">false</span>, <span class="keyword">null</span>, -<span class="number">1</span>);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先Vert.x会检查Event Bus是否已经启动，并且确保传入的地址不为空。然后Vert.x会传入一大堆参数创建一个新的<code>HandlerRegistration</code>类型的实例，并返回。可以推测<code>HandlerRegistration</code>是<code>MessageConsumer</code>接口的具体实现，它一定非常重要。所以我们来看一看<code>HandlerRegistration</code>类是个啥玩意。首先看一下<code>HandlerRegistration</code>的类体系结构：</p>
<p><img src="/assets/blog-img-k1/vertx-handlerregistration-ch.png" alt=""></p>
<p>可以看到<code>HandlerRegistration</code>类同时继承了<code>MessageConsumer&lt;T&gt;</code>以及<code>Handler&lt;Message&lt;T&gt;&gt;</code>接口，从其类名可以看出它相当于一个”Handler注册记录”，是非常重要的一个类。它有一堆的成员变量，构造函数对<code>vertx</code>, <code>metrics</code>, <code>eventBus</code>, <code>address</code>（发送地址）, <code>repliedAddress</code>（回复地址）, <code>localOnly</code>（是否在集群内传播）, <code>asyncResultHandler</code>等几个成员变量进行初始化，并且检查超时时间<code>timeout</code>，如果设定了超时时间那么设定并保存超时计时器（仅用于reply handler中），如果计时器时间到，代表回复超时。因为有一些函数还没介绍，超时的逻辑我们后边再讲。</p>
<blockquote>
<p>Note: 由于<code>MessageConsumer</code>接口继承了<code>ReadStream</code>接口，因此它支持back-pressure，其实现就在<code>HandlerRegistration</code>类中。我们将稍后解析back-pressure的实现。</p>
</blockquote>
<p>现在回到<code>consumer</code>方法中来。创建了<code>MessageConsumer</code>实例后，我们接着调用它的<code>handler</code>方法绑定上对应的消息处理函数。<code>handler</code>方法的实现位于<code>HandlerRegistration</code>类中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">synchronized</span> MessageConsumer&lt;T&gt; <span class="title">handler</span><span class="params">(Handler&lt;Message&lt;T&gt;&gt; handler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.handler = handler;</div><div class="line">  <span class="keyword">if</span> (<span class="keyword">this</span>.handler != <span class="keyword">null</span> &amp;&amp; !registered) &#123;</div><div class="line">    registered = <span class="keyword">true</span>;</div><div class="line">    eventBus.addRegistration(address, <span class="keyword">this</span>, repliedAddress != <span class="keyword">null</span>, localOnly);</div><div class="line">  &#125; <span class="keyword">else</span> <span class="keyword">if</span> (<span class="keyword">this</span>.handler == <span class="keyword">null</span> &amp;&amp; registered) &#123;</div><div class="line">    <span class="comment">// This will set registered to false</span></div><div class="line">    <span class="keyword">this</span>.unregister();</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先，<code>handler</code>方法将此<code>HandlerRegistration</code>中的<code>handler</code>成员设置为传入的消息处理函数。<code>HandlerRegistration</code>类中有一个<code>registered</code>标志位代表是否已绑定消息处理函数。<code>handler</code>方法会检查传入的<code>handler</code>是否为空且是否已绑定消息处理函数。如果不为空且未绑定，Vert.x就会将<code>registered</code>标志位设为<code>true</code>并且调用<code>eventBus</code>的<code>addRegistration</code>方法将此consumer注册至Event Bus上；如果<code>handler</code>为空且已绑定消息处理函数，我们就调用<code>unregister</code>方法注销当前的consumer。我们稍后会分析<code>unregister</code>方法的实现。</p>
<p>前面提到过注册consumer的逻辑位于Event Bus的<code>addRegistration</code>方法中，因此我们来分析一下它的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">addRegistration</span><span class="params">(String address, HandlerRegistration&lt;T&gt; registration,</span></span></div><div class="line">                                   <span class="keyword">boolean</span> replyHandler, <span class="keyword">boolean</span> localOnly) &#123;</div><div class="line">  Objects.requireNonNull(registration.getHandler(), <span class="string">"handler"</span>);</div><div class="line">  <span class="keyword">boolean</span> newAddress = addLocalRegistration(address, registration, replyHandler, localOnly);</div><div class="line">  addRegistration(newAddress, address, replyHandler, localOnly, registration::setResult);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>addRegistration</code>方法接受四个参数：发送地址<code>address</code>、传入的consumer <code>registration</code>、代表是否为reply handler的标志位<code>replyHandler</code>以及代表是否在集群范围内传播的标志位<code>localOnly</code>。首先确保传入的<code>HandlerRegistration</code>不为空。然后Vert.x会调用<code>addLocalRegistration</code>方法将此consumer注册至Event Bus上：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">boolean</span> <span class="title">addLocalRegistration</span><span class="params">(String address, HandlerRegistration&lt;T&gt; registration,</span></span></div><div class="line">                                           <span class="keyword">boolean</span> replyHandler, <span class="keyword">boolean</span> localOnly) &#123;</div><div class="line">  Objects.requireNonNull(address, <span class="string">"address"</span>);</div><div class="line"></div><div class="line">  Context context = Vertx.currentContext();</div><div class="line">  <span class="keyword">boolean</span> hasContext = context != <span class="keyword">null</span>;</div><div class="line">  <span class="keyword">if</span> (!hasContext) &#123;</div><div class="line">    <span class="comment">// Embedded</span></div><div class="line">    context = vertx.getOrCreateContext();</div><div class="line">  &#125;</div><div class="line">  registration.setHandlerContext(context);</div><div class="line"></div><div class="line">  <span class="keyword">boolean</span> newAddress = <span class="keyword">false</span>;</div><div class="line"></div><div class="line">  HandlerHolder holder = <span class="keyword">new</span> HandlerHolder&lt;&gt;(metrics, registration, replyHandler, localOnly, context);</div><div class="line"></div><div class="line">  Handlers handlers = handlerMap.get(address);</div><div class="line">  <span class="keyword">if</span> (handlers == <span class="keyword">null</span>) &#123;</div><div class="line">    handlers = <span class="keyword">new</span> Handlers();</div><div class="line">    Handlers prevHandlers = handlerMap.putIfAbsent(address, handlers);</div><div class="line">    <span class="keyword">if</span> (prevHandlers != <span class="keyword">null</span>) &#123;</div><div class="line">      handlers = prevHandlers;</div><div class="line">    &#125;</div><div class="line">    newAddress = <span class="keyword">true</span>;</div><div class="line">  &#125;</div><div class="line">  handlers.list.add(holder);</div><div class="line"></div><div class="line">  <span class="keyword">if</span> (hasContext) &#123;</div><div class="line">    HandlerEntry entry = <span class="keyword">new</span> HandlerEntry&lt;&gt;(address, registration);</div><div class="line">    context.addCloseHook(entry);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="keyword">return</span> newAddress;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先该方法要确保地址<code>address</code>不为空，接着它会获取当前线程下对应的Vert.x Context，如果获取不到则表明当前不在<code>Verticle</code>中（即Embedded），需要调用<code>vertx.getOrCreateContext()</code>来获取<code>Context</code>；然后将获取到的<code>Context</code>赋值给<code>registration</code>内部的<code>handlerContext</code>（代表消息处理对应的Vert.x Context）。</p>
<p>下面就要将给定的<code>registration</code>注册至Event Bus上了。这里Vert.x用一个<code>HandlerHolder</code>类来包装<code>registration</code>和<code>context</code>。接着Vert.x会从存储消息处理<code>Handler</code>的哈希表<code>handlerMap</code>中获取给定地址对应的<code>Handlers</code>，哈希表的类型为<code>ConcurrentMap&lt;String, Handlers&gt;</code>，key为地址，value为对应的<code>HandlerHolder</code>集合。注意这里的<code>Handlers</code>类代表一些<code>Handler</code>的集合，它内部维护着一个列表<code>list</code>用于存储每个<code>HandlerHolder</code>。<code>Handlers</code>类中只有一个<code>choose</code>函数，此函数根据轮询算法从<code>HandlerHolder</code>集合中选定一个<code>HandlerHolder</code>，这即是Event Bus发送消息时实现load-balancing的方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">Handlers</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">final</span> AtomicInteger pos = <span class="keyword">new</span> AtomicInteger(<span class="number">0</span>);</div><div class="line">  <span class="keyword">public</span> <span class="keyword">final</span> List&lt;HandlerHolder&gt; list = <span class="keyword">new</span> CopyOnWriteArrayList&lt;&gt;();</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> HandlerHolder <span class="title">choose</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">while</span> (<span class="keyword">true</span>) &#123;</div><div class="line">      <span class="keyword">int</span> size = list.size();</div><div class="line">      <span class="keyword">if</span> (size == <span class="number">0</span>) &#123;</div><div class="line">        <span class="keyword">return</span> <span class="keyword">null</span>;</div><div class="line">      &#125;</div><div class="line">      <span class="keyword">int</span> p = pos.getAndIncrement();</div><div class="line">      <span class="keyword">if</span> (p &gt;= size - <span class="number">1</span>) &#123;</div><div class="line">        pos.set(<span class="number">0</span>);</div><div class="line">      &#125;</div><div class="line">      <span class="keyword">try</span> &#123;</div><div class="line">        <span class="keyword">return</span> list.get(p);</div><div class="line">      &#125; <span class="keyword">catch</span> (IndexOutOfBoundsException e) &#123;</div><div class="line">        <span class="comment">// Can happen</span></div><div class="line">        pos.set(<span class="number">0</span>);</div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>获取到对应的<code>handlers</code>以后，Vert.x首先需要检查其是否为空，如果为空代表此地址还没有注册消息处理<code>Handler</code>，Vert.x就会创建一个<code>Handlers</code>并且将其置入<code>handlerMap</code>中，将<code>newAddress</code>标志位设为<code>true</code>代表这是一个新注册的地址，然后将其赋值给<code>handlers</code>。接着我们向<code>handlers</code>中的<code>HandlerHolder</code>列表<code>list</code>中添加刚刚创建的<code>HandlerHolder</code>实例，这样就将<code>registration</code>注册至Event Bus中了。</p>
<p>前面判断当前线程是否在Vert.x Context的标志位<code>hasContext</code>还有一个用途：如果当前线程在Vert.x Context下（比如在Verticle中），Vert.x会通过<code>addCloseHook</code>方法给当前的<code>context</code>添加一个钩子函数用于注销当前绑定的<code>registration</code>。当对应的<code>Verticle</code>被undeploy的时候，此Verticle绑定的所有消息处理<code>Handler</code>都会被unregister。Hook的类型为<code>HandlerEntry&lt;T&gt;</code>，它继承了<code>Closeable</code>接口，对应的逻辑在<code>close</code>函数中实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">HandlerEntry</span>&lt;<span class="title">T</span>&gt; <span class="keyword">implements</span> <span class="title">Closeable</span> </span>&#123;</div><div class="line">  <span class="keyword">final</span> String address;</div><div class="line">  <span class="keyword">final</span> HandlerRegistration&lt;T&gt; handler;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">HandlerEntry</span><span class="params">(String address, HandlerRegistration&lt;T&gt; handler)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.address = address;</div><div class="line">    <span class="keyword">this</span>.handler = handler;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="comment">// ...</span></div><div class="line"></div><div class="line">  <span class="comment">// Called by context on undeploy</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">close</span><span class="params">(Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler)</span> </span>&#123;</div><div class="line">    handler.unregister(completionHandler);</div><div class="line">    completionHandler.handle(Future.succeededFuture());</div><div class="line">  &#125;</div><div class="line"></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到<code>close</code>函数会将绑定的<code>registration</code>从Event Bus的<code>handlerMap</code>中移除并执行<code>completionHandler</code>中的逻辑，<code>completionHandler</code>可由用户指定。</p>
<p>那么在哪里调用这些绑定的hook呢？答案是在<code>DeploymentManager</code>类中的<code>doUndeploy</code>方法中，通过<code>context</code>的<code>runCloseHooks</code>方法执行绑定的hook函数。相关代码如下（只截取相关逻辑）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">doUndeploy</span><span class="params">(ContextImpl undeployingContext, Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler)</span> </span>&#123;</div><div class="line">  <span class="comment">// 前面代码略</span></div><div class="line">  context.runCloseHooks(ar2 -&gt; &#123;</div><div class="line"></div><div class="line">    <span class="keyword">if</span> (ar2.failed()) &#123;</div><div class="line">      <span class="comment">// Log error but we report success anyway</span></div><div class="line">      log.error(<span class="string">"Failed to run close hook"</span>, ar2.cause());</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">if</span> (ar.succeeded() &amp;&amp; undeployCount.incrementAndGet() == numToUndeploy) &#123;</div><div class="line">      reportSuccess(<span class="keyword">null</span>, undeployingContext, completionHandler);</div><div class="line">    &#125; <span class="keyword">else</span> <span class="keyword">if</span> (ar.failed() &amp;&amp; !failureReported.get()) &#123;</div><div class="line">      failureReported.set(<span class="keyword">true</span>);</div><div class="line">      reportFailure(ar.cause(), undeployingContext, completionHandler);</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">  <span class="comment">// 后面代码略</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>再回到<code>addRegistration</code>方法中。刚才<code>addLocalRegistration</code>方法的返回值<code>newAddress</code>代表对应的地址是否为新注册的。接着我们调用另一个版本的<code>addRegistration</code>方法，传入了一大堆参数：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">addRegistration</span><span class="params">(<span class="keyword">boolean</span> newAddress, String address,</span></span></div><div class="line">                                   <span class="keyword">boolean</span> replyHandler, <span class="keyword">boolean</span> localOnly,</div><div class="line">                                   Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler) &#123;</div><div class="line">  completionHandler.handle(Future.succeededFuture());</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>好吧，传入的前几个参数没用到。。。最后一个参数<code>completionHandler</code>传入的是<code>registration::setResult</code>方法引用，也就是说这个方法调用了对应<code>registration</code>的<code>setResult</code>方法。其实现位于<code>HandlerRegistration</code>类中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">setResult</span><span class="params">(AsyncResult&lt;Void&gt; result)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.result = result;</div><div class="line">  <span class="keyword">if</span> (completionHandler != <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="keyword">if</span> (result.succeeded()) &#123;</div><div class="line">      metric = metrics.handlerRegistered(address, repliedAddress);</div><div class="line">    &#125;</div><div class="line">    Handler&lt;AsyncResult&lt;Void&gt;&gt; callback = completionHandler;</div><div class="line">    vertx.runOnContext(v -&gt; callback.handle(result));</div><div class="line">  &#125; <span class="keyword">else</span> <span class="keyword">if</span> (result.failed()) &#123;</div><div class="line">    log.error(<span class="string">"Failed to propagate registration for handler "</span> + handler + <span class="string">" and address "</span> + address);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    metric = metrics.handlerRegistered(address, repliedAddress);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先先设置<code>registration</code>内部的<code>result</code>成员（正常情况下为<code>Future.succeededFuture()</code>）。接着Vert.x会判断<code>registration</code>是否绑定了<code>completionHandler</code>（与之前的<code>completionHandler</code>不同，这里的<code>completionHandler</code>是<code>MessageConsumer</code>注册成功时调用的<code>Handler</code>），若绑定则记录Metrics信息(<code>handlerRegistered</code>)并在Vert.x Context内调用<code>completionHandler</code>的逻辑；若未绑定<code>completionHandler</code>则仅记录Metrics信息。</p>
<p>到此为止，<code>consumer</code>方法的逻辑就分析完了。在分析<code>send</code>和<code>publish</code>方法的逻辑之前，我们先来看一下如何注销绑定的<code>MessageConsumer</code>。</p>
<h1 id="unregister">unregister</h1><p>我们通过调用<code>MessageConsumer</code>的<code>unregister</code>方法实现注销操作。Vert.x提供了两个版本的<code>unregister</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">void</span> <span class="title">unregister</span><span class="params">()</span></span>;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">void</span> <span class="title">unregister</span><span class="params">(Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler)</span></span>;</div></pre></td></tr></table></figure>
<p>其中第二个版本的<code>unregister</code>方法会在注销操作完成时调用传入的<code>completionHandler</code>。比如在cluster范围内注销consumer需要消耗一定的时间在集群内传播，因此第二个版本的方法就会派上用场。我们来看一下其实现，它们最后都是调用了<code>HandlerRegistration</code>类的<code>doUnregister</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">doUnregister</span><span class="params">(Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler, <span class="keyword">boolean</span> callEndHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (timeoutID != -<span class="number">1</span>) &#123;</div><div class="line">    vertx.cancelTimer(timeoutID);</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">if</span> (endHandler != <span class="keyword">null</span> &amp;&amp; callEndHandler) &#123;</div><div class="line">    Handler&lt;Void&gt; theEndHandler = endHandler;</div><div class="line">    Handler&lt;AsyncResult&lt;Void&gt;&gt; handler = completionHandler;</div><div class="line">    completionHandler = ar -&gt; &#123;</div><div class="line">      theEndHandler.handle(<span class="keyword">null</span>);</div><div class="line">      <span class="keyword">if</span> (handler != <span class="keyword">null</span>) &#123;</div><div class="line">        handler.handle(ar);</div><div class="line">      &#125;</div><div class="line">    &#125;;</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">if</span> (registered) &#123;</div><div class="line">    registered = <span class="keyword">false</span>;</div><div class="line">    eventBus.removeRegistration(address, <span class="keyword">this</span>, completionHandler);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    callCompletionHandlerAsync(completionHandler);</div><div class="line">  &#125;</div><div class="line">  registered = <span class="keyword">false</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>如果设定了超时定时器(<code>timeoutID</code>合法)，那么Vert.x会首先将定时器关闭。接着Vert.x会判断是否需要调用<code>endHandler</code>。那么<code>endHandler</code>又是什么呢？前面我们提到过<code>MessageConsumer</code>接口继承了<code>ReadStream</code>接口，而<code>ReadStream</code>接口定义了一个<code>endHandler</code>方法用于绑定一个<code>endHandler</code>，当stream中的数据读取完毕时会调用。而在Event Bus中，消息源源不断地从一处发送至另一处，因此只有在某个consumer<br>被unregister的时候，其对应的stream才可以叫“读取完毕”，因此Vert.x选择在<code>doUnregister</code>方法中调用<code>endHandler</code>。</p>
<p>接着Vert.x会判断此consumer是否已注册消息处理函数<code>Handler</code>（通过检查<code>registered</code>标志位），若已注册则将对应的<code>Handler</code>从Event Bus中的<code>handlerMap</code>中移除并将<code>registered</code>设为<code>false</code>；若还未注册<code>Handler</code>且提供了注销结束时的回调<code>completionHandler</code>(注意不是<code>HandlerRegistration</code>类的成员变量<code>completionHandler</code>，而是之前第二个版本的<code>unregister</code>中传入的<code>Handler</code>，用同样的名字很容易混。。。)，则通过<code>callCompletionHandlerAsync</code>方法调用回调函数。</p>
<p>从Event Bus中移除<code>Handler</code>的逻辑位于<code>EventBusImpl</code>类的<code>removeRegistration</code>方法中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">removeRegistration</span><span class="params">(String address, HandlerRegistration&lt;T&gt; handler, Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler)</span> </span>&#123;</div><div class="line">  HandlerHolder holder = removeLocalRegistration(address, handler);</div><div class="line">  removeRegistration(holder, address, completionHandler);</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">removeRegistration</span><span class="params">(HandlerHolder handlerHolder, String address,</span></span></div><div class="line">                                      Handler&lt;AsyncResult&lt;Void&gt;&gt; completionHandler) &#123;</div><div class="line">  callCompletionHandlerAsync(completionHandler);</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function">HandlerHolder <span class="title">removeLocalRegistration</span><span class="params">(String address, HandlerRegistration&lt;T&gt; handler)</span> </span>&#123;</div><div class="line">  Handlers handlers = handlerMap.get(address);</div><div class="line">  HandlerHolder lastHolder = <span class="keyword">null</span>;</div><div class="line">  <span class="keyword">if</span> (handlers != <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="keyword">synchronized</span> (handlers) &#123;</div><div class="line">      <span class="keyword">int</span> size = handlers.list.size();</div><div class="line">      <span class="comment">// Requires a list traversal. This is tricky to optimise since we can't use a set since</span></div><div class="line">      <span class="comment">// we need fast ordered traversal for the round robin</span></div><div class="line">      <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; size; i++) &#123;</div><div class="line">        HandlerHolder holder = handlers.list.get(i);</div><div class="line">        <span class="keyword">if</span> (holder.getHandler() == handler) &#123;</div><div class="line">          handlers.list.remove(i);</div><div class="line">          holder.setRemoved();</div><div class="line">          <span class="keyword">if</span> (handlers.list.isEmpty()) &#123;</div><div class="line">            handlerMap.remove(address);</div><div class="line">            lastHolder = holder;</div><div class="line">          &#125;</div><div class="line">          holder.getContext().removeCloseHook(<span class="keyword">new</span> HandlerEntry&lt;&gt;(address, holder.getHandler()));</div><div class="line">          <span class="keyword">break</span>;</div><div class="line">        &#125;</div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">return</span> lastHolder;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>其真正的<code>unregister</code>逻辑位于<code>removeLocalRegistration</code>方法中。首先需要从<code>handlerMap</code>中获取地址对应的<code>Handlers</code>实例<code>handlers</code>，如果<code>handlers</code>不为空，为了防止并发问题，Vert.x需要对其加锁后再进行操作。Vert.x需要遍历<code>handlers</code>中的列表，遇到与传入的<code>HandlerRegistration</code>相匹配的<code>HandlerHolder</code>就将其从列表中移除，然后调用对应<code>holder</code>的<code>setRemoved</code>方法标记其为已注销并记录Metrics数据（<code>handlerUnregistered</code>）。如果移除此<code>HandlerHolder</code>后<code>handlers</code>没有任何注册的<code>Handler</code>了，就将该地址对应的<code>Handlers</code>实例从<code>handlerMap</code>中移除并保存刚刚移除的<code>HandlerHolder</code>。另外，由于已经将此consumer注销，在undeploy verticle的时候不需要再进行unregister，因此这里还要将之前注册到context的hook移除。</p>
<p>调用完<code>removeLocalRegistration</code>方法以后，Vert.x会调用另一个版本的<code>removeRegistration</code>方法，调用<code>completionHandler</code>（用户在第二个版本的<code>unregister</code>方法中传入的处理函数）对应的逻辑，其它的参数都没什么用。。。</p>
<p>这就是<code>MessageConsumer</code>注销的逻辑实现。下面就到了本文的另一重头戏了 —— 发送消息相关的函数<code>send</code>和<code>publish</code>。</p>
<h1 id="send_&amp;_publish">send &amp; publish</h1><p><code>send</code>和<code>publish</code>的逻辑相近，只不过一个是发送至目标地址的某一消费者，一个是发布至目标地址的所有消费者。Vert.x使用一个标志位<code>send</code>来代表是否为点对点发送模式。</p>
<p>几个版本的<code>send</code>和<code>publish</code>最终都归结于生成消息对象然后调用<code>sendOrPubInternal</code>方法执行逻辑，只不过<code>send</code>标志位不同：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> &lt;T&gt; <span class="function">EventBus <span class="title">send</span><span class="params">(String address, Object message, DeliveryOptions options, Handler&lt;AsyncResult&lt;Message&lt;T&gt;&gt;&gt; replyHandler)</span> </span>&#123;</div><div class="line">  sendOrPubInternal(createMessage(<span class="keyword">true</span>, address, options.getHeaders(), message, options.getCodecName()), options, replyHandler);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> EventBus <span class="title">publish</span><span class="params">(String address, Object message, DeliveryOptions options)</span> </span>&#123;</div><div class="line">  sendOrPubInternal(createMessage(<span class="keyword">false</span>, address, options.getHeaders(), message, options.getCodecName()), options, <span class="keyword">null</span>);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>两个方法中都是通过<code>createMessage</code>方法来生成对应的消息对象的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">protected</span> MessageImpl <span class="title">createMessage</span><span class="params">(<span class="keyword">boolean</span> send, String address, MultiMap headers, Object body, String codecName)</span> </span>&#123;</div><div class="line">  Objects.requireNonNull(address, <span class="string">"no null address accepted"</span>);</div><div class="line">  MessageCodec codec = codecManager.lookupCodec(body, codecName);</div><div class="line">  <span class="meta">@SuppressWarnings</span>(<span class="string">"unchecked"</span>)</div><div class="line">  MessageImpl msg = <span class="keyword">new</span> MessageImpl(address, <span class="keyword">null</span>, headers, body, codec, send, <span class="keyword">this</span>);</div><div class="line">  <span class="keyword">return</span> msg;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>createMessage</code>方法接受5个参数：<code>send</code>即上面提到的标志位，<code>address</code>为发送目标地址，<code>headers</code>为设置的header，<code>body</code>代表发送的对象，<code>codecName</code>代表对应的Codec（消息编码解码器）名称。<code>createMessage</code>方法首先会确保地址不为空，然后通过<code>codecManager</code>来获取对应的<code>MessageCodec</code>。如果没有提供Codec(即<code>codecName</code>为空)，那么<code>codecManager</code>会根据发送对象<code>body</code>的类型来提供内置的Codec实现（具体逻辑请见<a href="https://github.com/eclipse/vert.x/blob/master/src/main/java/io/vertx/core/eventbus/impl/CodecManager.java#L47" target="_blank" rel="external">此处</a>）。准备好<code>MessageCodec</code>后，<code>createMessage</code>方法就会创建一个<code>MessageImpl</code>实例并且返回。</p>
<p>这里我们还需要了解一下<code>MessageImpl</code>的构造函数：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="title">MessageImpl</span><span class="params">(String address, String replyAddress, MultiMap headers, U sentBody,</span></span></div><div class="line">                   MessageCodec&lt;U, V&gt; messageCodec,</div><div class="line">                   <span class="keyword">boolean</span> send, EventBusImpl bus) &#123;</div><div class="line">  <span class="keyword">this</span>.messageCodec = messageCodec; <span class="comment">// Codec</span></div><div class="line">  <span class="keyword">this</span>.address = address; <span class="comment">// 发送目标地址</span></div><div class="line">  <span class="keyword">this</span>.replyAddress = replyAddress; <span class="comment">// 回复地址</span></div><div class="line">  <span class="keyword">this</span>.headers = headers; <span class="comment">// header</span></div><div class="line">  <span class="keyword">this</span>.sentBody = sentBody; <span class="comment">// 发送的对象</span></div><div class="line">  <span class="keyword">this</span>.send = send; <span class="comment">// 是否为点对点模式</span></div><div class="line">  <span class="keyword">this</span>.bus = bus; <span class="comment">// 相关的Event Bus实例</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>createMessage</code>方法并没有设置回复地址<code>replyAddress</code>。如果用户指定了<code>replyHandler</code>的话，后边<code>sendOrPubInternal</code>方法会对此消息实体进行加工，设置<code>replyAddress</code>并生成回复逻辑对应的<code>HandlerRegistration</code>。</p>
<p>我们看一下<code>sendOrPubInternal</code>方法的源码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">sendOrPubInternal</span><span class="params">(MessageImpl message, DeliveryOptions options,</span></span></div><div class="line">                                   Handler&lt;AsyncResult&lt;Message&lt;T&gt;&gt;&gt; replyHandler) &#123;</div><div class="line">  checkStarted();</div><div class="line">  HandlerRegistration&lt;T&gt; replyHandlerRegistration = createReplyHandlerRegistration(message, options, replyHandler);</div><div class="line">  SendContextImpl&lt;T&gt; sendContext = <span class="keyword">new</span> SendContextImpl&lt;&gt;(message, options, replyHandlerRegistration);</div><div class="line">  sendContext.next();</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>它接受三个参数：要发送的消息<code>message</code>，发送配置选项<code>options</code>以及回复处理函数<code>replyHandler</code>。首先<code>sendOrPubInternal</code>方法要检查Event Bus是否已启动，接着如果绑定了回复处理函数，Vert.x就会调用<code>createReplyHandlerRegistration</code>方法给消息实体<code>message</code>包装上回复地址，并且生成对应的reply consumer。接着Vert.x创建了一个包装消息的<code>SendContextImpl</code>实例并调用了其<code>next</code>方法。</p>
<p>我们一步一步来解释。首先是<code>createReplyHandlerRegistration</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> &lt;T&gt; <span class="function">HandlerRegistration&lt;T&gt; <span class="title">createReplyHandlerRegistration</span><span class="params">(MessageImpl message, DeliveryOptions options, Handler&lt;AsyncResult&lt;Message&lt;T&gt;&gt;&gt; replyHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (replyHandler != <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="keyword">long</span> timeout = options.getSendTimeout();</div><div class="line">    String replyAddress = generateReplyAddress();</div><div class="line">    message.setReplyAddress(replyAddress);</div><div class="line">    Handler&lt;Message&lt;T&gt;&gt; simpleReplyHandler = convertHandler(replyHandler);</div><div class="line">    HandlerRegistration&lt;T&gt; registration =</div><div class="line">      <span class="keyword">new</span> HandlerRegistration&lt;&gt;(vertx, metrics, <span class="keyword">this</span>, replyAddress, message.address, <span class="keyword">true</span>, replyHandler, timeout);</div><div class="line">    registration.handler(simpleReplyHandler);</div><div class="line">    <span class="keyword">return</span> registration;</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">null</span>;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>createReplyHandlerRegistration</code>方法首先检查传入的<code>replyHandler</code>是否为空（是否绑定了<code>replyHandler</code>，回复处理函数），如果为空则代表不需要处理回复，直接返回<code>null</code>；若<code>replyHandler</code>不为空，<code>createReplyHandlerRegistration</code>方法就会从配置中获取reply的最大超时时长(默认30s)，然后调用<code>generateReplyAddress</code>方法生成对应的回复地址<code>replyAddress</code>：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">final</span> AtomicLong replySequence = <span class="keyword">new</span> AtomicLong(<span class="number">0</span>);</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">protected</span> String <span class="title">generateReplyAddress</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">return</span> Long.toString(replySequence.incrementAndGet());</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>生成回复地址的逻辑有点简单。。。。<code>EventBusImpl</code>实例中维护着一个<code>AtomicLong</code>类型的<code>replySequence</code>成员变量代表对应的回复地址。每次生成的时候都会使其自增，然后转化为String。也就是说生成的<code>replyAddress</code>都类似于”1”、”5”这样，而不是我们想象中的直接回复至sender的地址。。。</p>
<p>生成完毕以后，<code>createReplyHandlerRegistration</code>方法会将生成的<code>replyAddress</code>设定到消息对象<code>message</code>中。接着Vert.x会通过<code>convertHandler</code>方法对<code>replyHandler</code>进行包装处理并生成类型简化为<code>Handler&lt;Message&lt;T&gt;&gt;</code>的<code>simpleReplyHandler</code>，它用于绑定至后面创建的reply consumer上。接着Vert.x会创建对应的reply consumer。关于<code>reply</code>操作的实现，我们后边会详细讲述。下面Vert.x就通过<code>handler</code>方法将生成的回复处理函数<code>simpleReplyHandler</code>绑定至创建好的reply consumer中，其底层实现我们之前已经分析过了，这里就不再赘述。最后此方法返回生成的<code>registration</code>，即对应的reply consumer。注意这个reply consumer是<strong>一次性</strong>的，也就是说Vert.x会在其接收到回复或超时的时候自动对其进行注销。</p>
<p>OK，现在回到<code>sendOrPubInternal</code>方法中来。下面Vert.x会创建一个<code>SendContextImpl</code>实例并调用其<code>next</code>方法。<code>SendContextImpl</code>类实现了<code>SendContext</code>接口，它相当于一个消息的封装体，并且可以与Event Bus中的<code>interceptors</code>（拦截器）结合使用。</p>
<p><code>SendContext</code>接口定义了三个方法：</p>
<ul>
<li><code>message</code>: 获取当前<code>SendContext</code>包装的消息实体</li>
<li><code>next</code>: 调用下一个消息拦截器</li>
<li><code>send</code>: 代表消息的发送模式是否为点对点模式</li>
</ul>
<p>在Event Bus中，消息拦截器本质上是一个<code>Handler&lt;SendContext&gt;</code>类型的处理函数。Event Bus内部存储着一个<code>interceptors</code>列表用于存储绑定的消息拦截器。我们可以通过<code>addInterceptor</code>和<code>removeInterceptor</code>方法进行消息拦截器的添加和删除操作。如果要进行链式拦截，则在每个拦截器中都应该调用对应<code>SendContext</code>的<code>next</code>方法，比如：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">eventBus.addInterceptor(sc -&gt; &#123;</div><div class="line">  <span class="comment">// 一些处理逻辑</span></div><div class="line">  sc.next(); <span class="comment">// 调用下一个拦截器</span></div><div class="line">&#125;);</div></pre></td></tr></table></figure>
<p>我们来看一下<code>SendContextImpl</code>类中<code>next</code>方法的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> <span class="class"><span class="keyword">class</span> <span class="title">SendContextImpl</span>&lt;<span class="title">T</span>&gt; <span class="keyword">implements</span> <span class="title">SendContext</span>&lt;<span class="title">T</span>&gt; </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">public</span> <span class="keyword">final</span> MessageImpl message;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">final</span> DeliveryOptions options;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">final</span> HandlerRegistration&lt;T&gt; handlerRegistration;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">final</span> Iterator&lt;Handler&lt;SendContext&gt;&gt; iter;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">SendContextImpl</span><span class="params">(MessageImpl message, DeliveryOptions options, HandlerRegistration&lt;T&gt; handlerRegistration)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.message = message;</div><div class="line">    <span class="keyword">this</span>.options = options;</div><div class="line">    <span class="keyword">this</span>.handlerRegistration = handlerRegistration;</div><div class="line">    <span class="keyword">this</span>.iter = interceptors.iterator();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> Message&lt;T&gt; <span class="title">message</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> message;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">next</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">if</span> (iter.hasNext()) &#123;</div><div class="line">      Handler&lt;SendContext&gt; handler = iter.next();</div><div class="line">      <span class="keyword">try</span> &#123;</div><div class="line">        handler.handle(<span class="keyword">this</span>);</div><div class="line">      &#125; <span class="keyword">catch</span> (Throwable t) &#123;</div><div class="line">        log.error(<span class="string">"Failure in interceptor"</span>, t);</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      sendOrPub(<span class="keyword">this</span>);</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">send</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> message.send();</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们可以看到，<code>SendContextImpl</code>类中维护了一个拦截器列表对应的迭代器。每次调用<code>next</code>方法时，如果迭代器中存在拦截器，就将下个拦截器取出并进行相关调用。如果迭代器为空，则代表拦截器都已经调用完毕，Vert.x就会调用<code>EventBusImpl</code>类下的<code>sendOrPub</code>方法进行消息的发送操作。</p>
<p><code>sendOrPub</code>方法仅仅在metrics模块中记录相关数据(<code>messageSent</code>)，最后调用<code>deliverMessageLocally(SendContextImpl&lt;T&gt;)</code>方法执行消息的发送逻辑:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">deliverMessageLocally</span><span class="params">(SendContextImpl&lt;T&gt; sendContext)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (!deliverMessageLocally(sendContext.message)) &#123;</div><div class="line">    <span class="comment">// no handlers</span></div><div class="line">    metrics.replyFailure(sendContext.message.address, ReplyFailure.NO_HANDLERS);</div><div class="line">    <span class="keyword">if</span> (sendContext.handlerRegistration != <span class="keyword">null</span>) &#123;</div><div class="line">      sendContext.handlerRegistration.sendAsyncResultFailure(ReplyFailure.NO_HANDLERS, <span class="string">"No handlers for address "</span></div><div class="line">                                                             + sendContext.message.address);</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里面又套了一层。。。它最后其实是调用了<code>deliverMessageLocally(MessageImpl)</code>方法。此方法返回值代表发送消息的目标地址是否注册有<code>MessageConsumer</code>，如果没有(<code>false</code>)则记录错误并调用<code>sendContext</code>中保存的回复处理函数处理错误（如果绑定了<code>replyHandler</code>的话）。</p>
<p><code>deliverMessageLocally(MessageImpl)</code>方法是真正区分<code>send</code>和<code>publish</code>的地方，我们来看一下其实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">boolean</span> <span class="title">deliverMessageLocally</span><span class="params">(MessageImpl msg)</span> </span>&#123;</div><div class="line">  msg.setBus(<span class="keyword">this</span>);</div><div class="line">  Handlers handlers = handlerMap.get(msg.address());</div><div class="line">  <span class="keyword">if</span> (handlers != <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="keyword">if</span> (msg.send()) &#123;</div><div class="line">      <span class="comment">//Choose one</span></div><div class="line">      HandlerHolder holder = handlers.choose();</div><div class="line">      metrics.messageReceived(msg.address(), !msg.send(), isMessageLocal(msg), holder != <span class="keyword">null</span> ? <span class="number">1</span> : <span class="number">0</span>);</div><div class="line">      <span class="keyword">if</span> (holder != <span class="keyword">null</span>) &#123;</div><div class="line">        deliverToHandler(msg, holder);</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="comment">// Publish</span></div><div class="line">      metrics.messageReceived(msg.address(), !msg.send(), isMessageLocal(msg), handlers.list.size());</div><div class="line">      <span class="keyword">for</span> (HandlerHolder holder: handlers.list) &#123;</div><div class="line">        deliverToHandler(msg, holder);</div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">true</span>;</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    metrics.messageReceived(msg.address(), !msg.send(), isMessageLocal(msg), <span class="number">0</span>);</div><div class="line">    <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先Vert.x需要从<code>handlerMap</code>中获取目标地址对应的处理函数集合<code>handlers</code>。接着，如果<code>handlers</code>不为空的话，Vert.x就会判断消息实体的<code>send</code>标志位。如果<code>send</code>标志位为<code>true</code>则代表以点对点模式发送，Vert.x就会通过<code>handlers</code>的<code>choose</code>方法(之前提到过)，按照轮询算法来获取其中的某一个<code>HandlerHolder</code>。获取到<code>HandlerHolder</code>之后，Vert.x会通过<code>deliverToHandler</code>方法将消息分发至<code>HandlerHolder</code>中进行处理；如果<code>send</code>标志位为<code>false</code>则代表向所有消费者发布消息，Vert.x就会对<code>handlers</code>中的每一个<code>HandlerHolder</code>依次调用<code>deliverToHandler</code>方法，以便将消息分发至所有注册到此地址的<code>Handler</code>中进行处理。</p>
<p>消息处理的真正逻辑就在<code>deliverToHandler</code>方法中。我们来看一下它的源码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">deliverToHandler</span><span class="params">(MessageImpl msg, HandlerHolder&lt;T&gt; holder)</span> </span>&#123;</div><div class="line">  <span class="comment">// Each handler gets a fresh copy</span></div><div class="line">  <span class="meta">@SuppressWarnings</span>(<span class="string">"unchecked"</span>)</div><div class="line">  Message&lt;T&gt; copied = msg.copyBeforeReceive();</div><div class="line"></div><div class="line">  <span class="keyword">if</span> (metrics.isEnabled()) &#123;</div><div class="line">    metrics.scheduleMessage(holder.getHandler().getMetric(), msg.isLocal());</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  holder.getContext().runOnContext((v) -&gt; &#123;</div><div class="line">    <span class="comment">// Need to check handler is still there - the handler might have been removed after the message were sent but</span></div><div class="line">    <span class="comment">// before it was received</span></div><div class="line">    <span class="keyword">try</span> &#123;</div><div class="line">      <span class="keyword">if</span> (!holder.isRemoved()) &#123;</div><div class="line">        holder.getHandler().handle(copied);</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">finally</span> &#123;</div><div class="line">      <span class="keyword">if</span> (holder.isReplyHandler()) &#123;</div><div class="line">        holder.getHandler().unregister();</div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先<code>deliverToHandler</code>方法会复制一份要发送的消息，然后<code>deliverToHandler</code>方法会调用<code>metrics</code>的<code>scheduleMessage</code>方法记录对应的Metrics信息（计划对消息进行处理。此函数修复了<a href="https://github.com/eclipse/vert.x/issues/1480" target="_blank" rel="external">Issue 1480</a>）。接着<code>deliverToHandler</code>方法会从传入的<code>HandlerHolder</code>中获取对应的Vert.x Context，然后调用<code>runOnContext</code>方法以便可以让消息处理逻辑在Vert.x Context中执行。为防止对应的handler在处理之前被移除，这里还需要检查一下<code>holder</code>的<code>isRemoved</code>属性。如果没有移除，那么就从<code>holder</code>中获取对应的<code>handler</code>并调用其<code>handle</code>方法执行消息的处理逻辑。注意这里获取的<code>handler</code>实际上是一个<code>HandlerRegistration</code>。前面提到过<code>HandlerRegistration</code>类同时实现了<code>MessageConsumer</code>接口和<code>Handler</code>接口，因此它兼具这两个接口所期望的功能。另外，之前我们提到过Vert.x会自动注销接收过回复的reply consumer，其逻辑就在这个finally块中。Vert.x会检查<code>holder</code>中的<code>handler</code>是否为reply handler（reply consumer)，如果是的话就调用其<code>unregister</code>方法将其注销，来确保reply consumer为一次性的。</p>
<p>之前我们提到过<code>MessageConsumer</code>继承了<code>ReadStream</code>接口，因此<code>HandlerRegistration</code>需要实现flow control(back-pressure)的相关逻辑。那么如何实现呢？我们看到,<code>HandlerRegistration</code>类中有一个<code>paused</code>标志位代表是否还继续处理消息。<code>ReadStream</code>接口中定义了两个函数用于控制stream的通断：当处理速度小于读取速度(发生拥塞)的时候我们可以通过<code>pause</code>方法暂停消息的传递，将积压的消息暂存于内部的消息队列（缓冲区）<code>pending</code>中；当相对速度正常的时候，我们可以通过<code>resume</code>方法恢复消息的传递和处理。</p>
<p>我们看一下<code>HandlerRegistration</code>中<code>handle</code>方法的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">handle</span><span class="params">(Message&lt;T&gt; message)</span> </span>&#123;</div><div class="line">  Handler&lt;Message&lt;T&gt;&gt; theHandler;</div><div class="line">  <span class="keyword">synchronized</span> (<span class="keyword">this</span>) &#123;</div><div class="line">    <span class="keyword">if</span> (paused) &#123;</div><div class="line">      <span class="keyword">if</span> (pending.size() &lt; maxBufferedMessages) &#123;</div><div class="line">        pending.add(message);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="keyword">if</span> (discardHandler != <span class="keyword">null</span>) &#123;</div><div class="line">          discardHandler.handle(message);</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">          log.warn(<span class="string">"Discarding message as more than "</span> + maxBufferedMessages + <span class="string">" buffered in paused consumer"</span>);</div><div class="line">        &#125;</div><div class="line">      &#125;</div><div class="line">      <span class="keyword">return</span>;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">if</span> (pending.size() &gt; <span class="number">0</span>) &#123;</div><div class="line">        pending.add(message);</div><div class="line">        message = pending.poll();</div><div class="line">      &#125;</div><div class="line">      theHandler = handler;</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">  deliver(theHandler, message);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>果然。。。<code>handle</code>方法在处理消息的基础上实现了拥塞控制的功能。为了防止资源争用，需要对自身进行加锁；首先<code>handle</code>方法会判断当前的<code>consumer</code>是否为<code>paused</code>状态，如果为<code>paused</code>状态，<code>handle</code>方法会检查当前缓冲区大小是否已经超过给定的最大缓冲区大小<code>maxBufferedMessages</code>，如果没超过，就将收到的消息push到缓冲区中；如果大于或等于阈值，Vert.x就需要丢弃超出的那部分消息。如果当前的<code>consumer</code>为正常状态，则如果缓冲区不为空，就将收到的消息push到缓冲区中并从缓冲区中pull队列首端的消息，然后调用<code>deliver</code>方法执行真正的消息处理逻辑。注意这里是在锁之外执行<code>deliver</code>方法的，这是为了保证在multithreaded worker context下可以并发传递消息（见<a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=473714" target="_blank" rel="external">Bug 473714
</a>）。由于multithreaded worker context允许在不同线程并发执行逻辑（见<a href="http://vertx.io/docs/vertx-core/java/#_multi_threaded_worker_verticles" target="_blank" rel="external">官方文档</a>），如果将<code>deliver</code>方法置于<code>synchronized</code>块之内，其他线程必须等待当前锁被释放才能进行消息的传递逻辑，因而不能做到“delivery concurrently”。</p>
<p><code>deliver</code>方法是真正执行“消息处理”逻辑的地方：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">deliver</span><span class="params">(Handler&lt;Message&lt;T&gt;&gt; theHandler, Message&lt;T&gt; message)</span> </span>&#123;</div><div class="line">  checkNextTick();</div><div class="line">  <span class="keyword">boolean</span> local = <span class="keyword">true</span>;</div><div class="line">  <span class="keyword">if</span> (message <span class="keyword">instanceof</span> ClusteredMessage) &#123;</div><div class="line">    <span class="comment">// A bit hacky</span></div><div class="line">    ClusteredMessage cmsg = (ClusteredMessage)message;</div><div class="line">    <span class="keyword">if</span> (cmsg.isFromWire()) &#123;</div><div class="line">      local = <span class="keyword">false</span>;</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">  String creditsAddress = message.headers().get(MessageProducerImpl.CREDIT_ADDRESS_HEADER_NAME);</div><div class="line">  <span class="keyword">if</span> (creditsAddress != <span class="keyword">null</span>) &#123;</div><div class="line">    eventBus.send(creditsAddress, <span class="number">1</span>);</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">try</span> &#123;</div><div class="line">    metrics.beginHandleMessage(metric, local);</div><div class="line">    theHandler.handle(message);</div><div class="line">    metrics.endHandleMessage(metric, <span class="keyword">null</span>);</div><div class="line">  &#125; <span class="keyword">catch</span> (Exception e) &#123;</div><div class="line">    log.error(<span class="string">"Failed to handleMessage"</span>, e);</div><div class="line">    metrics.endHandleMessage(metric, e);</div><div class="line">    <span class="keyword">throw</span> e;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先Vert.x会调用<code>checkNextTick</code>方法来检查消息队列（缓冲区）中是否存在更多的消息等待被处理，如果有的话就取出队列首端的消息并调用<code>deliver</code>方法将其传递给<code>handler</code>进行处理。这里仍需要注意并发问题，相关实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">checkNextTick</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (!pending.isEmpty()) &#123;</div><div class="line">    handlerContext.runOnContext(v -&gt; &#123;</div><div class="line">      Message&lt;T&gt; message;</div><div class="line">      Handler&lt;Message&lt;T&gt;&gt; theHandler;</div><div class="line">      <span class="keyword">synchronized</span> (HandlerRegistration.<span class="keyword">this</span>) &#123;</div><div class="line">        <span class="keyword">if</span> (paused || (message = pending.poll()) == <span class="keyword">null</span>) &#123;</div><div class="line">          <span class="keyword">return</span>;</div><div class="line">        &#125;</div><div class="line">        theHandler = handler;</div><div class="line">      &#125;</div><div class="line">      deliver(theHandler, message);</div><div class="line">    &#125;);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>检查完消息队列以后，Vert.x会接着根据<code>message</code>判断消息是否仅在本地进行处理并给<code>local</code>标志位赋值，<code>local</code>标志位将在记录Metrics数据时用到。</p>
<p>接下来我们看到Vert.x从消息的<code>headers</code>中获取了一个地址<code>creditsAddress</code>，如果<code>creditsAddress</code>存在就向此地址发送一条消息，body为<code>1</code>。那么这个<code>creditsAddress</code>又是啥呢？其实，它与flow control有关，我们会在下面详细分析。发送完<code>credit</code>消息以后，接下来就到了调用<code>handler</code>处理消息的时刻了。在处理消息之前需要调用<code>metrics</code>的<code>beginHandleMessage</code>方法记录消息开始处理的metrics数据，在处理完消息以后需要调用<code>endHandleMessage</code>方法记录消息处理结束的metrics数据。</p>
<p>嗯。。。到此为止，消息的发送和处理过程就已经一目了然了。下面我们讲一讲之前代码中出现的<code>creditsAddress</code>到底是啥玩意～</p>
<h1 id="MessageProducer">MessageProducer</h1><p>之前我们提到过，Vert.x定义了两个接口作为 <strong>flow control aware object</strong> 的规范：<code>WriteStream</code>以及<code>ReadStream</code>。对于<code>ReadStream</code>我们已经不再陌生了，<code>MessageConsumer</code>就继承了它；那么大家应该可以想象到，有<code>MessageConsumer</code>就必有<code>MessageProducer</code>。不错，Vert.x中的<code>MessageProducer</code>接口对应某个<code>address</code>上的消息生产者，同时它继承了<code>WriteStream</code>接口，因此<code>MessageProducer</code>的实现类<code>MessageProducerImpl</code>同样具有flow control的能力。我们可以把<code>MessageProducer</code>看做是一个具有flow control功能的增强版的<code>EventBus</code>。我们可以通过<code>EventBus</code>接口的<code>publisher</code>方法创建一个<code>MessageProducer</code>。</p>
<p>对<code>MessageProducer</code>有了初步了解之后，我们就可以解释前面<code>deliver</code>方法中的<code>creditsAddress</code>了。<code>MessageProducer</code>接口的实现类 —— <code>MessageProducerImpl</code>类的流量控制功能是基于<code>credit</code>的，其内部会维护一个<code>credit</code>值代表“发送消息的能力”，其默认值等于<code>DEFAULT_WRITE_QUEUE_MAX_SIZE</code>：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">int</span> maxSize = DEFAULT_WRITE_QUEUE_MAX_SIZE;</div><div class="line"><span class="keyword">private</span> <span class="keyword">int</span> credits = DEFAULT_WRITE_QUEUE_MAX_SIZE;</div></pre></td></tr></table></figure>
<p>在采用点对点模式发送消息的时候，<code>MessageProducer</code>底层会调用<code>doSend</code>方法进行消息的发送。发送依然利用Event Bus的<code>send</code>方法，只不过<code>doSend</code>方法中添加了flow control的相关逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">synchronized</span> &lt;R&gt; <span class="function"><span class="keyword">void</span> <span class="title">doSend</span><span class="params">(T data, Handler&lt;AsyncResult&lt;Message&lt;R&gt;&gt;&gt; replyHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (credits &gt; <span class="number">0</span>) &#123;</div><div class="line">    credits--;</div><div class="line">    <span class="keyword">if</span> (replyHandler == <span class="keyword">null</span>) &#123;</div><div class="line">      bus.send(address, data, options);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      bus.send(address, data, options, replyHandler);</div><div class="line">    &#125;</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    pending.add(data);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>与<code>MessageConsumer</code>类似，<code>MessageProducer</code>内部同样保存着一个消息队列（缓冲区）用于暂存堆积的消息。当<code>credits</code>大于<strong>0</strong>的时候代表可以发送消息（没有出现拥塞），Vert.x就会调用Event Bus的<code>send</code>方法进行消息的发送，同时<code>credits</code>要减1；如果<code>credits</code>小于等于0，则代表此时消息发送的速度太快，出现了拥塞，需要暂缓发送，因此将要发送的对象暂存于缓冲区中。大家可能会问，<code>credits</code>值不断减小，那么恢复消息发送能力（增大<code>credits</code>）的逻辑在哪呢？这就要提到<code>creditsAddress</code>了。我们看一下<code>MessageProducerImpl</code>类的构造函数：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="title">MessageProducerImpl</span><span class="params">(Vertx vertx, String address, <span class="keyword">boolean</span> send, DeliveryOptions options)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.vertx = vertx;</div><div class="line">  <span class="keyword">this</span>.bus = vertx.eventBus();</div><div class="line">  <span class="keyword">this</span>.address = address;</div><div class="line">  <span class="keyword">this</span>.send = send;</div><div class="line">  <span class="keyword">this</span>.options = options;</div><div class="line">  <span class="keyword">if</span> (send) &#123;</div><div class="line">    String creditAddress = UUID.randomUUID().toString() + <span class="string">"-credit"</span>;</div><div class="line">    creditConsumer = bus.consumer(creditAddress, msg -&gt; &#123;</div><div class="line">      doReceiveCredit(msg.body());</div><div class="line">    &#125;);</div><div class="line">    options.addHeader(CREDIT_ADDRESS_HEADER_NAME, creditAddress);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    creditConsumer = <span class="keyword">null</span>;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>MessageProducerImpl</code>的构造函数中生成了一个<code>creditAddress</code>，然后给该地址绑定了一个<code>Handler</code>，当收到消息时调用<code>doReceiveCredit</code>方法执行解除拥塞，恢复消息发送的逻辑。<code>MessageProducerImpl</code>会将此<code>MessageConsumer</code>保存，以便在关闭消息生产者流的时候将其注销。接着构造函数会往<code>options</code>的<code>headers</code>中添加一条记录，保存对应的<code>creditAddress</code>，这也就是上面我们在<code>deliver</code>函数中获取的<code>creditAddress</code>：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line"><span class="comment">// 位于HandlerRegistration类的deliver函数中</span></div><div class="line">String creditsAddress = message.headers().get(MessageProducerImpl.CREDIT_ADDRESS_HEADER_NAME);</div><div class="line"><span class="keyword">if</span> (creditsAddress != <span class="keyword">null</span>) &#123;</div><div class="line">  eventBus.send(creditsAddress, <span class="number">1</span>);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这样，发送消息到<code>creditsAddress</code>的逻辑也就好理解了。由于<code>deliver</code>函数的逻辑就是处理消息，因此这里向<code>creditsAddress</code>发送一个 <strong>1</strong> 其实就是将对应的<code>credits</code>值加1。恢复消息发送的逻辑位于<code>MessageProducerImpl</code>类的<code>doReceiveCredit</code>方法中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title">doReceiveCredit</span><span class="params">(<span class="keyword">int</span> credit)</span> </span>&#123;</div><div class="line">  credits += credit;</div><div class="line">  <span class="keyword">while</span> (credits &gt; <span class="number">0</span>) &#123;</div><div class="line">    T data = pending.poll();</div><div class="line">    <span class="keyword">if</span> (data == <span class="keyword">null</span>) &#123;</div><div class="line">      <span class="keyword">break</span>;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      credits--;</div><div class="line">      bus.send(address, data, options);</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">final</span> Handler&lt;Void&gt; theDrainHandler = drainHandler;</div><div class="line">  <span class="keyword">if</span> (theDrainHandler != <span class="keyword">null</span> &amp;&amp; credits &gt;= maxSize / <span class="number">2</span>) &#123;</div><div class="line">    <span class="keyword">this</span>.drainHandler = <span class="keyword">null</span>;</div><div class="line">    vertx.runOnContext(v -&gt; theDrainHandler.handle(<span class="keyword">null</span>));</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>逻辑一目了然。首先给<code>credits</code>加上发送过来的值（正常情况下为1），然后恢复发送能力，将缓冲区的数据依次取出、发送然后减小<code>credits</code>。同时如果<code>MessageProducer</code>绑定了<code>drainHandler</code>(消息流不拥塞的时候调用的逻辑，详见<a href="http://vertx.io/docs/vertx-core/java/#_writestream" target="_blank" rel="external">官方文档</a>)，并且<code>MessageProducer</code>发送的消息不再拥塞（<code>credits &gt;= maxSize / 2</code>），那么就在Vert.x Context中执行<code>drainHandler</code>中的逻辑。</p>
<p>怎么样，体会到Vert.x中flow control的强大之处了吧！官方文档中<code>MessageProducer</code>的篇幅几乎没有，只在介绍<code>WriteStream</code>的时候提了提，因此这部分也可以作为<code>MessageProducer</code>的参考。</p>
<h1 id="reply">reply</h1><p>最后就是消息的回复逻辑 —— <code>reply</code>方法了。<code>reply</code>方法的实现位于<code>MessageImpl</code>类中，最终调用的是<code>reply(Object, DeliveryOptions, Handler&lt;AsyncResult&lt;Message&lt;R&gt;&gt;&gt;)</code>这个版本：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> &lt;R&gt; <span class="function"><span class="keyword">void</span> <span class="title">reply</span><span class="params">(Object message, DeliveryOptions options, Handler&lt;AsyncResult&lt;Message&lt;R&gt;&gt;&gt; replyHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (replyAddress != <span class="keyword">null</span>) &#123;</div><div class="line">    sendReply(bus.createMessage(<span class="keyword">true</span>, replyAddress, options.getHeaders(), message, options.getCodecName()), options, replyHandler);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里<code>reply</code>方法同样调用<code>EventBus</code>的<code>createMessage</code>方法创建要回复的消息实体，传入的<code>replyAddress</code>即为之前讲过的生成的非常简单的回复地址。然后再将消息实体、配置以及对应的<code>replyHandler</code>（如果有的话）传入<code>sendReply</code>方法进行消息的回复。最后其实是调用了Event Bus中的四个参数的<code>sendReply</code>方法，它的逻辑与之前讲过的<code>sendOrPubInternal</code>非常相似：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">protected</span> &lt;T&gt; <span class="function"><span class="keyword">void</span> <span class="title">sendReply</span><span class="params">(MessageImpl replyMessage, MessageImpl replierMessage, DeliveryOptions options,</span></span></div><div class="line">                             Handler&lt;AsyncResult&lt;Message&lt;T&gt;&gt;&gt; replyHandler) &#123;</div><div class="line">  <span class="keyword">if</span> (replyMessage.address() == <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"address not specified"</span>);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    HandlerRegistration&lt;T&gt; replyHandlerRegistration = createReplyHandlerRegistration(replyMessage, options, replyHandler);</div><div class="line">    <span class="keyword">new</span> ReplySendContextImpl&lt;&gt;(replyMessage, options, replyHandlerRegistration, replierMessage).next();</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>参数中<code>replyMessage</code>代表回复消息实体，<code>replierMessage</code>则代表回复者自身的消息实体(sender)。</p>
<p>如果地址为空则抛出异常；如果地址不为空，则先调用<code>createReplyHandlerRegistration</code>方法创建对应的<code>replyHandlerRegistration</code>。<code>createReplyHandlerRegistration</code>方法的实现之前已经讲过了。注意这里的<code>createReplyHandlerRegistration</code>其实对应的是此replier的回复，因为Vert.x中的 <strong>Request-Response</strong> 消息模型不限制相互回复（通信）的次数。当然如果没有指定此replier的回复的<code>replyHandler</code>，那么此处的<code>replyHandlerRegistration</code>就为空。最后<code>sendReply</code>方法会创建一个<code>ReplySendContextImpl</code>并调用其<code>next</code>方法。</p>
<p><code>ReplySendContextImpl</code>类同样是<code>SendContext</code>接口的一个实现（继承了<code>SendContextImpl</code>类）。<code>ReplySendContextImpl</code>比起其父类就多保存了一个<code>replierMessage</code>。<code>next</code>方法的逻辑与父类逻辑非常相似，只不过将回复的逻辑替换成了另一个版本的<code>sendReply</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">next</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (iter.hasNext()) &#123;</div><div class="line">    Handler&lt;SendContext&gt; handler = iter.next();</div><div class="line">    handler.handle(<span class="keyword">this</span>);</div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    sendReply(<span class="keyword">this</span>, replierMessage);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>然而。。。<code>sendReply</code>方法并没有用到传入的<code>replierMessage</code>，所以这里最终还是调用了<code>sendOrPub</code>方法（尼玛，封装的<code>ReplySendContextImpl</code>貌似并没有什么卵用，可能为以后的扩展考虑？）。。。之后的逻辑我们都已经分析过了。</p>
<p>这里再强调一点。当我们发送消息同时指定<code>replyHandler</code>的时候，其内部为reply创建的reply consumer(类型为<code>HandlerRegistration</code>)指定了<code>timeout</code>。这个定时器从<code>HandlerRegistration</code>创建的时候就开始计时了。我们回顾一下：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">if</span> (timeout != -<span class="number">1</span>) &#123;</div><div class="line">  timeoutID = vertx.setTimer(timeout, tid -&gt; &#123;</div><div class="line">    metrics.replyFailure(address, ReplyFailure.TIMEOUT);</div><div class="line">    sendAsyncResultFailure(ReplyFailure.TIMEOUT, <span class="string">"Timed out after waiting "</span> + timeout + <span class="string">"(ms) for a reply. address: "</span> + address);</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>计时器会在超时的时候记录错误并强制注销当前consumer。由于reply consumer是<strong>一次性的</strong>，当收到reply的时候，Vert.x会自动对reply consumer调用<code>unregister</code>方法对其进行注销（实现位于<code>EventBusImpl#deliverToHandler</code>方法中），而在注销逻辑中会关闭定时器（参见前面对<code>doUnregister</code>方法的解析）；如果超时，那么计时器就会触发，Vert.x会调用<code>sendAsyncResultFailure</code>方法注销当前reply consumer并处理错误。</p>
<h1 id="synchronized的性能问题">synchronized的性能问题</h1><p>大家可能看到为了防止race condition，Vert.x底层大量使用了<code>synchronized</code>关键字（重量级锁）。这会不会影响性能呢？其实，如果开发者遵循Vert.x的线程模型和开发规范（使用Verticle）的话，有些地方的<code>synchronized</code>对应的锁会被优化为 <strong>偏向锁</strong> 或 <strong>轻量级锁</strong>（因为通常都是同一个Event Loop线程获取对应的锁），这样性能总体开销不会太大。当然如果使用Multi-threaded worker verticles就要格外关注性能问题了。。。</p>
<h1 id="总结">总结</h1><p>我们来简略地总结一下Event Bus的工作原理。当我们调用<code>consumer</code>绑定一个<code>MessageConsumer</code>时，Vert.x会将它保存至Event Bus实例内部的Map中；当我们通过<code>send</code>或<code>publish</code>向对应的地址发送消息的时候，Vert.x会遍历Event Bus中存储consumer的Map，获取与地址相对应的consumer集合，然后根据相应的策略传递并处理消息(<code>send</code>通过轮询策略获取任意一个consumer并将消息传递至consumer中，<code>publish</code>则会将消息传递至所有注册到对应地址的consumer中)。同时，<code>MessageConsumer</code>和<code>MessageProducer</code>这两个接口的实现都具有flow control功能，因此它们也可以用在<code>Pump</code>中。</p>
<p>Event Bus是Vert.x中最为重要的一部分之一，探索Event Bus的源码可以加深我们对Event Bus工作原理的理解。作为开发者，只会使用框架是不够的，能够理解内部的实现原理和精华，并对其进行改进才是更为重要的。本篇文章分析的是Local模式下的Event Bus，下篇文章我们将来探索一下生产环境中更常用的 <strong>Clustered Event Bus</strong> 的实现原理，敬请期待！</p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;Event Bus是Vert.x的“神经系统”，是最为关键的几个部分之一。今天我们就来探索一下Event Bus的实现原理。本篇分析的是Local模式的Event Bus，对应的Vert.x版本为&lt;strong&gt;3.3.2&lt;/strong&gt;。&lt;/p&gt;
&lt;p&gt;本文假定读者有一
    
    </summary>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/categories/Vert-x/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
      <category term="源码分析" scheme="http://www.sczyh30.com/tags/%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90/"/>
    
  </entry>
  
  <entry>
    <title>最近真忙～</title>
    <link href="http://www.sczyh30.com/posts/Life/busy-enough-2016-08/"/>
    <id>http://www.sczyh30.com/posts/Life/busy-enough-2016-08/</id>
    <published>2016-08-29T16:25:00.000Z</published>
    <updated>2018-09-23T14:15:44.000Z</updated>
    
    <content type="html"><![CDATA[<p>最近越来越忙了，要忙各种项目，Blog都没空更了。。。等着抽空更新一波之前没发的文章。。。</p>
<p><img src="/assets/blog-img-k1/CoYX-A0XEAEYpu9.jpg" alt=""></p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;最近越来越忙了，要忙各种项目，Blog都没空更了。。。等着抽空更新一波之前没发的文章。。。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;/assets/blog-img-k1/CoYX-A0XEAEYpu9.jpg&quot; alt=&quot;&quot;&gt;&lt;/p&gt;

    
    </summary>
    
      <category term="Life" scheme="http://www.sczyh30.com/categories/Life/"/>
    
    
      <category term="生活" scheme="http://www.sczyh30.com/tags/%E7%94%9F%E6%B4%BB/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x Blueprint 系列教程(三) | Micro Shop 微服务实战</title>
    <link href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-3-micro-shop-microservice/"/>
    <id>http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-3-micro-shop-microservice/</id>
    <published>2016-08-29T10:00:00.000Z</published>
    <updated>2016-11-24T06:07:28.000Z</updated>
    
    <content type="html"><![CDATA[<p>Vert.x 蓝图项目已经发布至Vert.x官方网站：<a href="http://vertx.io/blog/vert-x-blueprint-tutorials/" target="_blank" rel="external">Vert.x Blueprint Tutorials</a></p>
<hr>
<p><strong>Vert.x 蓝图系列</strong> 的第三篇教程出炉咯！这篇教程是微服务实战相关的主题。篇幅较长，team给了模板用于渲染对应的文档，因此这里就直接放链接了：</p>
<ul>
<li><a href="http://sczyh30.github.io/vertx-blueprint-microservice/cn/index.html" target="_blank" rel="external">Vert.x 蓝图 - Micro Shop 微服务实战 (开发篇)</a></li>
<li><a href="http://sczyh30.github.io/vertx-blueprint-microservice/cn/api-gateway.html" target="_blank" rel="external">Vert.x 蓝图 - Micro Shop 微服务实战 (API Gateway)</a></li>
</ul>
<p>对应的GitHub Repository:  <a href="https://github.com/sczyh30/vertx-blueprint-microservice" target="_blank" rel="external">sczyh30/vertx-blueprint-microservice</a></p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;Vert.x 蓝图项目已经发布至Vert.x官方网站：&lt;a href=&quot;http://vertx.io/blog/vert-x-blueprint-tutorials/&quot; target=&quot;_blank&quot; rel=&quot;external&quot;&gt;Vert.x Blueprint Tut
    
    </summary>
    
      <category term="Asynchronous" scheme="http://www.sczyh30.com/categories/Asynchronous/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="微服务" scheme="http://www.sczyh30.com/tags/%E5%BE%AE%E6%9C%8D%E5%8A%A1/"/>
    
      <category term="分布式系统" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x Blueprint 系列教程(二) | Vert.x Kue 教程（Web部分）</title>
    <link href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-http-tutorial/"/>
    <id>http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-http-tutorial/</id>
    <published>2016-07-24T07:00:00.000Z</published>
    <updated>2016-11-24T06:07:28.000Z</updated>
    
    <content type="html"><![CDATA[<p><a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-core-tutorial/">上部分蓝图教程</a>中我们一起探索了如何用Vert.x开发一个基于消息的应用。在这部分教程中，我们将粗略地探索一下<code>kue-http</code>模块的实现。</p>
<h1 id="Vert-x_Kue_REST_API">Vert.x Kue REST API</h1><p><code>kue-http</code>模块中只有一个类<code>KueHttpVerticle</code>，作为整个REST API以及UI服务的实现。对REST API部分来说，如果看过我们之前的 <a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-1-todo-backend-tutorial/">Vert.x 蓝图 | 待办事项服务开发教程</a> 的话，你应该对这一部分非常熟悉了，因此这里我们就不详细解释了。有关使用Vert.x Web实现REST API的教程可参考 <a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-1-todo-backend-tutorial/">Vert.x 蓝图 | 待办事项服务开发教程</a>。</p>
<h1 id="将Kue_UI与Vert-x_Web进行适配">将Kue UI与Vert.x Web进行适配</h1><p>除了REST API之外，我们还给Vert.x Kue提供了一个用户界面。我们复用了Automattic/Kue的用户界面所以我们就不用写前端代码了（部分API有变动的地方我已进行了修改）。我们只需要将前端代码与Vert.x Web适配即可。</p>
<p>首先，前端的代码都属于静态资源，因此我们需要配置路由来允许访问静态资源：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">router.route().handler(StaticHandler.create(root));</div></pre></td></tr></table></figure>
<p>这样我们就可以直接访问静态资源咯～</p>
<p>注意到Kue UI使用了<strong>Jade</strong>（最近貌似改名叫Pug了）作为模板引擎，因此我们需要一个Jade模板解析器。好在Vert.x Web提供了一个Jade模板解析的实现: <code>io.vertx:vertx-web-templ-jade</code>，所以我们可以利用这个实现来渲染UI。首先在类中定义一个<code>JadeTemplateEngine</code>并在<code>start</code>方法中初始化：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">engine = JadeTemplateEngine.create();</div></pre></td></tr></table></figure>
<p>然后我们就可以写一个处理器方法来根据不同的任务状态来渲染UI：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">render</span><span class="params">(RoutingContext context, String state)</span> </span>&#123;</div><div class="line">  <span class="keyword">final</span> String uiPath = <span class="string">"webroot/views/job/list.jade"</span>; <span class="comment">// (1)</span></div><div class="line">  String title = config().getString(<span class="string">"kue.ui.title"</span>, <span class="string">"Vert.x Kue"</span>);</div><div class="line">  kue.getAllTypes()</div><div class="line">    .setHandler(resultHandler(context, r -&gt; &#123;</div><div class="line">      context.put(<span class="string">"state"</span>, state) <span class="comment">// (2)</span></div><div class="line">        .put(<span class="string">"types"</span>, r)</div><div class="line">        .put(<span class="string">"title"</span>, title);</div><div class="line">      engine.render(context, uiPath, res -&gt; &#123; <span class="comment">// (3)</span></div><div class="line">        <span class="keyword">if</span> (res.succeeded()) &#123;</div><div class="line">          context.response()</div><div class="line">            .putHeader(<span class="string">"content-type"</span>, <span class="string">"text/html"</span>) <span class="comment">// (4)</span></div><div class="line">            .end(res.result());</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">          context.fail(res.cause());</div><div class="line">        &#125;</div><div class="line">      &#125;);</div><div class="line">    &#125;));</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们需要给渲染引擎指定我们前端代码的地址 (1)。然后我们从Redis中获取其中所有的任务类型，然后向解析器context中添加任务状态、网页标题、任务类型等信息供渲染器渲染使用 (2)。接着我们就可以调用<code>engine.render(context, path, handler)</code>方法进行渲染 (3)。如果渲染成功，我们将页面写入HTTP Response (4)。</p>
<p>现在我们可以利用<code>render</code>方法去实现其它的路由函数了：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleUIActive</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  render(context, <span class="string">"active"</span>);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>然后我们给它绑个路由就可以了：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">router.route(KUE_UI_ACTIVE).handler(<span class="keyword">this</span>::handleUIActive);</div></pre></td></tr></table></figure>
<p>是不是非常方便呢？不仅如此，Vert.x Web还提供了其它各种模板引擎的支持，比如 <em>FreeMaker</em>, <em>Pebble</em> 以及 <em>Thymeleaf 3</em>。如果感兴趣的话，你可以查阅<a href="http://vertx.io/docs/vertx-web/java/#_templates" target="_blank" rel="external">官方文档</a>来获取详细的使用指南。</p>
<h1 id="展示时间！">展示时间！</h1><p>是不是等不及要看UI长啥样了？现在我们就来展示一下！首先构建项目：</p>
<pre><code><span class="title">gradle</span> build
</code></pre><p><code>kue-http</code>需要<code>kue-core</code>运行着（因为<code>kue-core</code>里注册了Event Bus服务），因此我们先运行<code>kue-core</code>，再运行<code>kue-http</code>。不要忘记运行Redis:</p>
<pre><code>redis-server
java -jar kue-core/build/libs/vertx-blueprint-kue-core<span class="class">.jar</span> -cluster -ha -conf config/config<span class="class">.json</span>
java -jar kue-http/build/libs/vertx-blueprint-kue-http<span class="class">.jar</span> -cluster -ha -conf config/config.json
</code></pre><p>为了更好地观察任务处理的流程，我们再运行一个示例：</p>
<pre><code>java -jar kue-example<span class="regexp">/build/</span>libs<span class="regexp">/vertx-blueprint-kue-example.jar -cluster -ha -conf config/</span>config.json
</code></pre><p>好啦！现在在浏览器中访问<code>http://localhost:8080</code>，我们的Kue UI就呈现在我们眼前啦！</p>
<p><img src="https://raw.githubusercontent.com/sczyh30/vertx-blueprint-job-queue/master/docs/images/vertx_kue_ui_1.png" alt="Vert.x Kue UI"></p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;&lt;a href=&quot;http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-core-tutorial/&quot;&gt;上部分蓝图教程&lt;/a&gt;中我们一起探索了如何用Vert.x开发一个基于消息的应用。在这
    
    </summary>
    
      <category term="Asynchronous" scheme="http://www.sczyh30.com/categories/Asynchronous/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
      <category term="Web开发" scheme="http://www.sczyh30.com/tags/Web%E5%BC%80%E5%8F%91/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x Blueprint 系列教程(二) | 开发基于消息的应用 - Vert.x Kue 教程（Core部分）</title>
    <link href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-core-tutorial/"/>
    <id>http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-core-tutorial/</id>
    <published>2016-07-24T06:00:00.000Z</published>
    <updated>2016-11-24T06:07:28.000Z</updated>
    
    <content type="html"><![CDATA[<p>本文章是 <strong>Vert.x 蓝图系列</strong> 的第二篇教程。全系列：</p>
<ul>
<li><a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-1-todo-backend-tutorial/">Vert.x Blueprint 系列教程(一) | 待办事项服务开发教程</a></li>
<li>Vert.x Blueprint 系列教程(二) | 开发基于消息的应用 - Vert.x Kue 教程</li>
<li><a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-3-micro-shop-microservice/">Vert.x Blueprint 系列教程(三) | Micro Shop 微服务应用实践</a></li>
</ul>
<h1 id="前言">前言</h1><p>欢迎回到Vert.x 蓝图系列～在本教程中，我们将利用Vert.x开发一个基于消息的应用 - Vert.x Kue，它是一个使用Vert.x开发的优先级工作队列，数据存储使用的是 <em>Redis</em> 。Vert.x Kue是<a href="https://github.com/Automattic/kue" target="_blank" rel="external">Automattic/kue</a>的Vert.x实现版本。我们可以使用Vert.x Kue来处理各种各样的任务，比如文件转换、订单处理等等。</p>
<p>通过本教程，你将会学习到以下内容：</p>
<ul>
<li>消息、消息系统以及事件驱动的运用</li>
<li>Vert.x <strong>Event Bus</strong> 的几种事件机制（发布/订阅、点对点模式）</li>
<li>设计 <strong>分布式</strong> 的Vert.x应用</li>
<li>工作队列的设计</li>
<li><strong>Vert.x Service Proxy</strong>（服务代理）的运用</li>
<li>更深层次的Redis运用</li>
</ul>
<p>本教程是 <a href="http://vertx.io/blog/vert-x-blueprint-tutorials/" target="_blank" rel="external">Vert.x 蓝图系列</a> 的第二篇教程，对应的Vert.x版本为 <strong>3.3.3</strong> 。本教程中的完整代码已托管至<a href="https://github.com/sczyh30/vertx-blueprint-job-queue/tree/master" target="_blank" rel="external">GitHub</a>。</p>
<h1 id="Vert-x的消息系统">Vert.x的消息系统</h1><p>既然我们要用Vert.x开发一个基于消息的应用，那么我们先来瞅一瞅Vert.x的消息系统吧～在Vert.x中，我们可以通过 <strong>Event Bus</strong> 来发送和接收各种各样的消息，这些消息可以来自不同的<code>Vertx</code>实例。怎么样，很酷吧？我们都将消息发送至Event Bus上的某个<strong>地址</strong>上，这个地址可以是任意的字符串。</p>
<p>Event Bus支持三种消息机制：<strong>发布/订阅</strong>(Publish/Subscribe)、<strong>点对点</strong>(Point to point)以及<strong>请求/回应</strong>(Request-Response)模式。下面我们就来看一看这几种机制。</p>
<h2 id="发布/订阅模式">发布/订阅模式</h2><p>在<strong>发布/订阅模式</strong>中，消息被发布到Event Bus的某一个地址上，所有订阅此地址的<code>Handler</code>都会接收到该消息并且调用相应的处理逻辑。我们来看一看示例代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div></pre></td><td class="code"><pre><div class="line">EventBus eventBus = vertx.eventBus();</div><div class="line"></div><div class="line">eventBus.consumer(<span class="string">"foo.bar.baz"</span>, r -&gt; &#123; <span class="comment">// subscribe to `foo.bar.baz` address</span></div><div class="line">  System.out.println(<span class="string">"1: "</span> + r.body());</div><div class="line">&#125;);</div><div class="line">eventBus.consumer(<span class="string">"foo.bar.baz"</span>, r -&gt; &#123; <span class="comment">// subscribe to `foo.bar.baz` address</span></div><div class="line">  System.out.println(<span class="string">"2: "</span> + r.body());</div><div class="line">&#125;);</div><div class="line"></div><div class="line">eventBus.publish(<span class="string">"foo.bar.baz"</span>, <span class="string">"+1s"</span>); <span class="comment">// 向此地址发送消息</span></div></pre></td></tr></table></figure>
<p>我们可以通过<code>vertx.eventBus()</code>方法获取<code>EventBus</code>的引用，然后我们就可以通过<code>consume</code>方法订阅某个地址的消息并且绑定一个<code>Handler</code>。接着我们通过<code>publish</code>向此地址发送消息。如果运行上面的例子，我们会得到一下结果：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">2: +1s</div><div class="line">1: +1s</div></pre></td></tr></table></figure>
<h2 id="点对点模式">点对点模式</h2><p>如果我们把上面的示例中的<code>publish</code>方法替代成<code>send</code>方法，上面的实例就变成<strong>点对点模式</strong>了。在点对点模式中，消息被发布到Event Bus的某一个地址上。Vert.x会将此消息传递给其中监听此地址的<code>Handler</code>之一。如果有多个<code>Handler</code>绑定到此地址，那么就使用轮询算法随机挑一个<code>Handler</code>传递消息。比如在此示例中，程序只会打印<code>2: +1s</code>或者<code>1: +1s</code>之中的一个。</p>
<h2 id="请求/回应模式">请求/回应模式</h2><p>当我们绑定的<code>Handler</code>接收到消息的时候，我们可不可以给消息的发送者回复呢？当然了！当我们通过<code>send</code>方法发送消息的时候，我们可以同时指定一个回复处理函数(reply handler)。然后当某个消息的订阅者接收到消息的时候，它就可以给发送者回复消息；如果发送者接收到了回复，发送者绑定的回复处理函数就会被调用。这就是<strong>请求/回应模式</strong>。</p>
<p>好啦，现在我们已经粗略了解了Vert.x中的消息系统 - Event Bus的基本使用，下面我们就看看Vert.x Kue的基本设计。有关更多关于Event Bus的信息请参考<a href="http://vertx.io/docs/vertx-core/java/#event_bus" target="_blank" rel="external">Vert.x Core Manual - Event Bus</a>。</p>
<h1 id="Vert-x_Kue_架构设计">Vert.x Kue 架构设计</h1><h2 id="Vert-x_Kue_组件划分">Vert.x Kue 组件划分</h2><p>在我们的项目中，我们将Vert.x Kue划分为两个模块：</p>
<ul>
<li><code>kue-core</code>: 核心组件，提供优先级队列的功能</li>
<li><code>kue-http</code>: Web组件，提供Web UI以及REST API</li>
</ul>
<p>另外我们还提供一个示例模块<code>kue-example</code>用于演示以及阐述如何使用Vert.x Kue。</p>
<p>既然我们的项目有两个模块，那么你一定会好奇：两个模块之间是如何进行通信的？并且如果我们写自己的Kue应用的话，我们该怎样去调用Kue Core中的服务呢？不要着急，谜底将在后边的章节中揭晓:-)</p>
<h2 id="Vert-x_Kue_核心模块">Vert.x Kue 核心模块</h2><p>回顾一下Vert.x Kue的作用 - 优先级工作队列，所以在Vert.x Kue的核心模块中我们设计了以下的类：</p>
<ul>
<li><code>Job</code> - 任务（作业）数据实体</li>
<li><code>JobService</code> - 异步服务接口，提供操作任务以及获取数据的相关逻辑</li>
<li><code>KueWorker</code> - 用于处理任务的Verticle</li>
<li><code>Kue</code> - 工作队列</li>
</ul>
<p>前边我们提到过，我们的两个组件之间需要一种通信机制可以互相通信 - 这里我们使用Vert.x的<strong>集群模式</strong>，即以clustered的模式来部署Verticle。这样的环境下的Event Bus同样也是集群模式的，因此各个组件可以通过集群模式下的Event Bus进行通信。很不错吧？在Vert.x的集群模式下，我们需要指定一个集群管理器<code>ClusterManager</code>。这里我们使用默认的<code>HazelcastClusterManager</code>，使用<strong>Hazelcast</strong>作为集群管理。</p>
<p>在Vert.x Kue中，我们将<code>JobService</code>服务发布至分布式的Event Bus上，这样其它的组件就可以通过Event Bus调用该服务了。我们设计了一个<code>KueVerticle</code>用于注册服务。Vert.x提供了Vert.x Service Proxy（服务代理组件），可以很方便地将服务注册至Event Bus上，然后在其它地方获取此服务的代理并调用。我们将在下面的章节中详细介绍<strong>Vert.x Service Proxy</strong>。</p>
<h2 id="基于Future的异步模式">基于Future的异步模式</h2><p>在我们的Vert.x Kue中，大多数的异步方法都是基于<code>Future</code>的。如果您看过蓝图系列的第一篇文章的话，您一定不会对这种模式很陌生。在Vert.x 3.3.2中，我们的<code>Future</code>支持基本的响应式的操作，比如<code>map</code>和<code>compose</code>。它们用起来非常方便，因为我们可以将多个<code>Future</code>以响应式的方式组合起来而不用担心陷入回调地狱中。</p>
<h2 id="Vert-x_Kue中的事件">Vert.x Kue中的事件</h2><p>正如我们在<a href="https://github.com/sczyh30/vertx-blueprint-job-queue/blob/master/docs/zh-cn/vertx-kue-features.zh-cn.md" target="_blank" rel="external">Vert.x Kue 特性介绍</a>中提到的那样，Vert.x Kue支持两种级别的事件：<strong>任务事件(job events)</strong> 以及 <strong>队列事件(queue events)</strong>。在Vert.x Kue中，我们设计了三种事件地址：</p>
<ul>
<li><code>vertx.kue.handler.job.{handlerType}.{addressId}.{jobType}</code>: 某个特定任务的任务事件地址</li>
<li><code>vertx.kue.handler.workers.{eventType}</code>: （全局）队列事件地址</li>
<li><code>vertx.kue.handler.workers.{eventType}.{addressId}</code>: 某个特定任务的内部事件地址</li>
</ul>
<p>在<a href="https://github.com/sczyh30/vertx-blueprint-job-queue/blob/master/docs/zh-cn/vertx-kue-features.zh-cn.md" target="_blank" rel="external">特性介绍文档</a>中，我们提到了以下几种任务事件：</p>
<ul>
<li><code>start</code> 开始处理一个任务 (<code>onStart</code>)</li>
<li><code>promotion</code> 一个延期的任务时间已到，提升至工作队列中 (<code>onPromotion</code>)</li>
<li><code>progress</code> 任务的进度变化 (<code>onProgress</code>)</li>
<li><code>failed_attempt</code> 任务处理失败，但是还可以重试 (<code>onFailureAttempt</code>)</li>
<li><code>failed</code> 任务处理失败并且不能重试 (<code>onFailure</code>)</li>
<li><code>complete</code> 任务完成 (<code>onComplete</code>)</li>
<li><code>remove</code> 任务从后端存储中移除 (<code>onRemove</code>)</li>
</ul>
<p>队列事件也相似，只不过需要加前缀<code>job_</code>。这些事件都会通过<code>send</code>方法发送至Event Bus上。每一个任务都有对应的任务事件地址，因此它们能够正确地接收到对应的事件并进行相应的处理逻辑。</p>
<p>特别地，我们还有两个内部事件：<code>done</code>和<code>done_fail</code>。<code>done</code>事件对应一个任务在底层的处理已经完成，而<code>done_fail</code>事件对应一个任务在底层的处理失败。这两个事件使用第三种地址进行传递。</p>
<h2 id="任务状态">任务状态</h2><p>在Vert.x Kue中，任务共有五种状态：</p>
<ul>
<li><code>INACTIVE</code>: 任务还未开始处理，在工作队列中等待处理</li>
<li><code>ACTIVE</code>: 任务正在处理中</li>
<li><code>COMPLETE</code>: 任务处理完成</li>
<li><code>FAILED</code>: 任务处理失败</li>
<li><code>DELAYED</code>: 任务延时处理，正在等待计时器时间到并提升至工作队列中</li>
</ul>
<p>我们使用状态图来描述任务状态的变化：</p>
<p><img src="https://raw.githubusercontent.com/sczyh30/vertx-blueprint-job-queue/master/docs/images/job_state_machine.png" alt="Job State Machine"></p>
<p>以及任务状态的变化伴随的事件：</p>
<p><img src="https://raw.githubusercontent.com/sczyh30/vertx-blueprint-job-queue/master/docs/images/event_emit_state_machine.png" alt="Events with state change"></p>
<h2 id="整体设计">整体设计</h2><p>为了让大家对Vert.x Kue的架构有大致的了解，我用一幅图来简略描述整个Vert.x Kue的设计：</p>
<p><img src="https://raw.githubusercontent.com/sczyh30/vertx-blueprint-job-queue/master/docs/images/kue_diagram.png" alt="Diagram - How Vert.x Kue works"></p>
<p>现在我们对Vert.x Kue的设计有了大致的了解了，下面我们就来看一看Vert.x Kue的代码实现了～</p>
<h1 id="项目结构">项目结构</h1><p>我们来开始探索Vert.x Kue的旅程吧！首先我们先从GitHub上clone源代码：</p>
<pre><code>git clone <span class="string">https:</span><span class="comment">//github.com/sczyh30/vertx-blueprint-job-queue.git</span>
</code></pre><p>然后你可以把项目作为Gradle项目导入你的IDE中。（如何导入请参考相关IDE帮助文档）</p>
<p>正如我们之前所提到的，我们的Vert.x Kue中有两个功能模块和一个实例模块，因此我们需要在Gradle工程文件中定义三个子工程。我们来看一下本项目中的<code>build.gradle</code>文件：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div><div class="line">68</div><div class="line">69</div><div class="line">70</div><div class="line">71</div><div class="line">72</div><div class="line">73</div><div class="line">74</div><div class="line">75</div><div class="line">76</div><div class="line">77</div><div class="line">78</div><div class="line">79</div><div class="line">80</div><div class="line">81</div><div class="line">82</div><div class="line">83</div><div class="line">84</div><div class="line">85</div><div class="line">86</div><div class="line">87</div><div class="line">88</div><div class="line">89</div><div class="line">90</div><div class="line">91</div><div class="line">92</div><div class="line">93</div><div class="line">94</div><div class="line">95</div><div class="line">96</div><div class="line">97</div><div class="line">98</div><div class="line">99</div><div class="line">100</div><div class="line">101</div><div class="line">102</div><div class="line">103</div><div class="line">104</div><div class="line">105</div><div class="line">106</div><div class="line">107</div><div class="line">108</div><div class="line">109</div></pre></td><td class="code"><pre><div class="line">configure(<span class="keyword">allprojects</span>) &#123; <span class="keyword">project</span> -&gt;</div><div class="line"></div><div class="line">  ext &#123;</div><div class="line">    vertxVersion = <span class="string">"3.3.2"</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  apply plugin: <span class="string">'java'</span></div><div class="line"></div><div class="line">  <span class="keyword">repositories</span> &#123;</div><div class="line">    jcenter()</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="keyword">dependencies</span> &#123;</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-core:$&#123;vertxVersion&#125;"</span>)</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-codegen:$&#123;vertxVersion&#125;"</span>)</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-rx-java:$&#123;vertxVersion&#125;"</span>)</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-hazelcast:$&#123;vertxVersion&#125;"</span>)</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-lang-ruby:$&#123;vertxVersion&#125;"</span>)</div><div class="line"></div><div class="line">    testCompile(<span class="string">"io.vertx:vertx-unit:$&#123;vertxVersion&#125;"</span>)</div><div class="line">    testCompile <span class="keyword">group</span>: <span class="string">'junit'</span>, name: <span class="string">'junit'</span>, version: <span class="string">'4.12'</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="keyword">sourceSets</span> &#123;</div><div class="line">    main &#123;</div><div class="line">      java &#123;</div><div class="line">        srcDirs += <span class="string">'src/main/generated'</span></div><div class="line">      &#125;</div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  compileJava &#123;</div><div class="line">    <span class="keyword">targetCompatibility</span> = <span class="number">1.8</span></div><div class="line">    <span class="keyword">sourceCompatibility</span> = <span class="number">1.8</span></div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">project</span>(<span class="string">"kue-core"</span>) &#123;</div><div class="line"></div><div class="line">  <span class="keyword">dependencies</span> &#123;</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-redis-client:$&#123;vertxVersion&#125;"</span>)</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-service-proxy:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  jar &#123;</div><div class="line">    archiveName = <span class="string">'vertx-blueprint-kue-core.jar'</span></div><div class="line">    <span class="keyword">from</span> &#123; <span class="keyword">configurations</span>.<span class="keyword">compile</span>.<span class="keyword">collect</span> &#123; it.isDirectory() ? it : zipTree(it) &#125; &#125;</div><div class="line">    manifest &#123;</div><div class="line">      attributes <span class="string">'Main-Class'</span>: <span class="string">'io.vertx.core.Launcher'</span></div><div class="line">      attributes <span class="string">'Main-Verticle'</span>: <span class="string">'io.vertx.blueprint.kue.queue.KueVerticle'</span></div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="keyword">task</span> annotationProcessing(type: JavaCompile, <span class="keyword">group</span>: <span class="string">'build'</span>) &#123; <span class="comment">// codegen</span></div><div class="line">    <span class="keyword">source</span> = <span class="keyword">sourceSets</span>.main.java</div><div class="line">    <span class="keyword">classpath</span> = <span class="keyword">configurations</span>.<span class="keyword">compile</span></div><div class="line">    <span class="keyword">destinationDir</span> = <span class="keyword">project</span>.<span class="keyword">file</span>(<span class="string">'src/main/generated'</span>)</div><div class="line">    <span class="keyword">options</span>.compilerArgs = [</div><div class="line">      <span class="string">"-proc:only"</span>,</div><div class="line">      <span class="string">"-processor"</span>, <span class="string">"io.vertx.codegen.CodeGenProcessor"</span>,</div><div class="line">      <span class="string">"-AoutputDirectory=$&#123;project.projectDir&#125;/src/main"</span></div><div class="line">    ]</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  compileJava &#123;</div><div class="line">    <span class="keyword">targetCompatibility</span> = <span class="number">1.8</span></div><div class="line">    <span class="keyword">sourceCompatibility</span> = <span class="number">1.8</span></div><div class="line"></div><div class="line">    dependsOn annotationProcessing</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">project</span>(<span class="string">"kue-http"</span>) &#123;</div><div class="line"></div><div class="line">  <span class="keyword">dependencies</span> &#123;</div><div class="line">    <span class="keyword">compile</span>(<span class="keyword">project</span>(<span class="string">":kue-core"</span>))</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-web:$&#123;vertxVersion&#125;"</span>)</div><div class="line">    <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-web-templ-jade:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  jar &#123;</div><div class="line">    archiveName = <span class="string">'vertx-blueprint-kue-http.jar'</span></div><div class="line">    <span class="keyword">from</span> &#123; <span class="keyword">configurations</span>.<span class="keyword">compile</span>.<span class="keyword">collect</span> &#123; it.isDirectory() ? it : zipTree(it) &#125; &#125;</div><div class="line">    manifest &#123;</div><div class="line">      attributes <span class="string">'Main-Class'</span>: <span class="string">'io.vertx.core.Launcher'</span></div><div class="line">      attributes <span class="string">'Main-Verticle'</span>: <span class="string">'io.vertx.blueprint.kue.http.KueHttpVerticle'</span></div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">project</span>(<span class="string">"kue-example"</span>) &#123;</div><div class="line"></div><div class="line">  <span class="keyword">dependencies</span> &#123;</div><div class="line">    <span class="keyword">compile</span>(<span class="keyword">project</span>(<span class="string">":kue-core"</span>))</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  jar &#123;</div><div class="line">    archiveName = <span class="string">'vertx-blueprint-kue-example.jar'</span></div><div class="line">    <span class="keyword">from</span> &#123; <span class="keyword">configurations</span>.<span class="keyword">compile</span>.<span class="keyword">collect</span> &#123; it.isDirectory() ? it : zipTree(it) &#125; &#125;</div><div class="line">    manifest &#123;</div><div class="line">      attributes <span class="string">'Main-Class'</span>: <span class="string">'io.vertx.core.Launcher'</span></div><div class="line">      attributes <span class="string">'Main-Verticle'</span>: <span class="string">'io.vertx.blueprint.kue.example.LearningVertxVerticle'</span></div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">task</span> wrapper(type: Wrapper) &#123;</div><div class="line">  gradleVersion = <span class="string">'2.12'</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>(⊙o⊙)…比之前的待办事项服务项目中的长不少诶。。。我们来解释一下：</p>
<ul>
<li>在<code>configure(allprojects)</code>作用域中，我们配置了一些全局信息（对所有子工程都适用）。</li>
<li>我们定义了三个子工程：<code>kue-core</code>、<code>kue-http</code>以及<code>kue-example</code>。这里我们来解释一下里面用到的依赖。在<code>kue-core</code>中，<code>vertx-redis-client</code>用于Redis通信，<code>vertx-service-proxy</code>用于Event Bus上的服务代理。在<code>kue-http</code>中，我们将<code>kue-core</code>子工程作为它的一个依赖。<code>vertx-web</code>和<code>vertx-web-templ-jade</code>用于Kue Web端的开发。</li>
<li>任务<code>annotationProcessing</code>用于注解处理（Vert.x Codegen）。我们已经在上一篇教程中介绍过了，这里就不展开讲了。</li>
</ul>
<p>我们还需要在 <code>settings.gradle</code> 中配置工程：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">rootProject.name = <span class="string">'vertx-blueprint-job-queue'</span></div><div class="line"></div><div class="line"><span class="keyword">include</span> <span class="string">"kue-core"</span></div><div class="line"><span class="keyword">include</span> <span class="string">"kue-http"</span></div><div class="line"><span class="keyword">include</span> <span class="string">"kue-example"</span></div></pre></td></tr></table></figure>
<p>看完了配置文件以后，我们再来浏览一下我们的项目目录结构：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div></pre></td><td class="code"><pre><div class="line">.</div><div class="line">├── build.gradle</div><div class="line">├── kue-core</div><div class="line">│   └── src</div><div class="line">│       ├── main</div><div class="line">│       │   ├── java</div><div class="line">│       │   └── resources</div><div class="line">│       └── test</div><div class="line">│           ├── java</div><div class="line">│           └── resources</div><div class="line">├── kue-example</div><div class="line">│   └── src</div><div class="line">│       ├── main</div><div class="line">│       │   ├── java</div><div class="line">│       │   └── resources</div><div class="line">│       └── test</div><div class="line">│           ├── java</div><div class="line">│           └── resources</div><div class="line">├── kue-http</div><div class="line">│   └── src</div><div class="line">│       ├── main</div><div class="line">│       │   ├── java</div><div class="line">│       │   └── resources</div><div class="line">│       └── test</div><div class="line">│           ├── java</div><div class="line">│           └── resources</div><div class="line">└── settings.gradle</div></pre></td></tr></table></figure>
<p>在Gradle中，项目的源码都位于<code>{projectName}/src/main/java</code>目录内。这篇教程是围绕Vert.x Kue Core的，所以我们的代码都在<code>kue-core</code>目录中。</p>
<p>好啦！现在我们已经对Vert.x Kue项目的整体结构有了大致的了解了，下面我们开始源码探索之旅！</p>
<h1 id="任务实体_-_不仅仅是一个数据对象">任务实体 - 不仅仅是一个数据对象</h1><p>Vert.x Kue是用来处理任务的，因此我们先来看一下代表任务实体的<code>Job</code>类。<code>Job</code>类位于<code>io.vertx.blueprint.kue.queue</code>包下。代码可能有点长，不要担心，我们把它分成几部分，分别来解析。</p>
<h2 id="任务成员属性">任务成员属性</h2><p>我们先来看一下<code>Job</code>类中的成员属性：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@DataObject</span>(generateConverter = <span class="keyword">true</span>)</div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">Job</span> </span>&#123;</div><div class="line">    <span class="comment">// job properties</span></div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">final</span> String address_id;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> id = -<span class="number">1</span>;</div><div class="line">    <span class="keyword">private</span> String zid;</div><div class="line">    <span class="keyword">private</span> String type;</div><div class="line">    <span class="keyword">private</span> JsonObject data;</div><div class="line">    <span class="keyword">private</span> Priority priority = Priority.NORMAL;</div><div class="line">    <span class="keyword">private</span> JobState state = JobState.INACTIVE;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> delay = <span class="number">0</span>;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">int</span> max_attempts = <span class="number">1</span>;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">boolean</span> removeOnComplete = <span class="keyword">false</span>;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">int</span> ttl = <span class="number">0</span>;</div><div class="line">    <span class="keyword">private</span> JsonObject backoff;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">int</span> attempts = <span class="number">0</span>;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">int</span> progress = <span class="number">0</span>;</div><div class="line">    <span class="keyword">private</span> JsonObject result;</div><div class="line"></div><div class="line">    <span class="comment">// job metrics</span></div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> created_at;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> promote_at;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> updated_at;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> failed_at;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> started_at;</div><div class="line">    <span class="keyword">private</span> <span class="keyword">long</span> duration;</div><div class="line"></div><div class="line"></div><div class="line">    <span class="comment">// ...</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我去。。。好多属性！我们一个一个地解释：</p>
<ul>
<li><code>address_id</code>: 一个UUID序列，作为Event Bus的地址</li>
<li><code>id</code>: 任务的编号(id)</li>
<li><code>type</code>: 任务的类型</li>
<li><code>data</code>: 任务携带的数据，以 <code>JsonObject</code> 类型表示</li>
<li><code>priority</code>: 任务优先级，以 <code>Priority</code> 枚举类型表示。默认优先级为正常(<code>NORMAL</code>)</li>
<li><code>delay</code>: 任务的延迟时间，默认是 <strong>0</strong></li>
<li><code>state</code>: 任务状态，以 <code>JobState</code> 枚举类型表示。默认状态为等待(<code>INACTIVE</code>)</li>
<li><code>attempts</code>: 任务已经尝试执行的次数</li>
<li><code>max_attempts</code>: 任务尝试执行次数的最大阈值</li>
<li><code>removeOnComplete</code>: 代表任务完成时是否自动从后台移除</li>
<li><code>zid</code>: <code>zset</code>操作对应的编号(zid)，保持先进先出顺序</li>
<li><code>ttl</code>: TTL(Time to live)</li>
<li><code>backoff</code>: 任务重试配置，以 <code>JsonObject</code> 类型表示</li>
<li><code>progress</code>: 任务执行的进度</li>
<li><code>result</code>: 任务执行的结果，以 <code>JsonObject</code> 类型表示</li>
</ul>
<p>还有这些统计数据：</p>
<ul>
<li><code>created_at</code>: 代表此任务创建的时间</li>
<li><code>promote_at</code>: 代表此任务从延时状态被提升至等待状态时的时间</li>
<li><code>updated_at</code>: 代表任务更新的时间</li>
<li><code>failed_at</code>: 代表任务失败的时间</li>
<li><code>started_at</code>: 代表任务开始的时间</li>
<li><code>duration</code>: 代表处理任务花费的时间，单位为毫秒(<code>ms</code>)</li>
</ul>
<p>你可能注意到在 <code>Job</code> 类中还存在着几个静态成员变量：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> Logger logger = LoggerFactory.getLogger(Job.class);</div><div class="line"></div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> Vertx vertx;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> RedisClient client;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> EventBus eventBus;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">setVertx</span><span class="params">(Vertx v, RedisClient redisClient)</span> </span>&#123;</div><div class="line">  vertx = v;</div><div class="line">  client = redisClient;</div><div class="line">  eventBus = vertx.eventBus();</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>对于 <code>logger</code> 对象，我想大家应该都很熟悉，它代表一个Vert.x Logger实例用于日志记录。但是你一定想问为什么 <code>Job</code> 类中存在着一个<code>Vertx</code>类型的静态成员。<code>Job</code>类不应该是一个数据对象吗？当然咯！<code>Job</code>类代表一个数据对象，但<strong>不仅仅</strong>是一个数据对象。这里我模仿了一些Automattic/kue的风格，把一些任务相关逻辑方法放到了<code>Job</code>类里，它们大多都是基于<code>Future</code>的异步方法，因此可以很方便地去调用以及进行组合变换。比如：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">job.save()</div><div class="line">    .compose(Job::updateNow)</div><div class="line">    .compose(j -&gt; j.log(<span class="string">"good!"</span>));</div></pre></td></tr></table></figure>
<p>由于我们不能在<code>Job</code>类被JVM加载的时候就获取<code>Vertx</code>实例，我们必须手动给<code>Job</code>类中的静态<code>Vertx</code>成员赋值。这里我们是在<code>Kue</code>类中对其进行赋值的。当我们创建一个工作队列的时候，<code>Job</code>类中的静态成员变量会被初始化。同时为了保证程序的正确性，我们需要一个方法来检测静态成员变量是否初始化。当我们在创建一个任务的时候，如果静态成员此时未被初始化，那么日志会给出警告：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">_checkStatic</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (vertx == <span class="keyword">null</span>) &#123;</div><div class="line">    logger.warn(<span class="string">"static Vertx instance in Job class is not initialized!"</span>);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们还注意到 <code>Job</code> 类也是由<code>@DataObject</code>注解修饰的。Vert.x Codegen可以处理含有<code>@DataObject</code>注解的类并生成对应的JSON转换器，并且Vert.x Service Proxy也需要数据对象。</p>
<p>在<code>Job</code>类中我们有四个构造函数。其中<code>address_id</code>成员必须在一个任务被创建时就被赋值，默认情况下此地址用一个唯一的UUID字符串表示。每一个构造函数中我们都要调用<code>_checkStatic</code>函数来检测静态成员变量是否被初始化。</p>
<h2 id="任务事件辅助函数">任务事件辅助函数</h2><p>正如我们之前所提到的那样，我们通过一个特定的地址<code>vertx.kue.handler.job.{handlerType}.{addressId}.{jobType}</code>在分布式的Event Bus上发送和接收任务事件(job events)。所以我们提供了两个用于发送和接收事件的辅助函数<code>emit</code>和<code>on</code>(类似于Node.js中的<code>EventEmitter</code>)：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="keyword">public</span> &lt;T&gt; <span class="function">Job <span class="title">on</span><span class="params">(String event, Handler&lt;Message&lt;T&gt;&gt; handler)</span> </span>&#123;</div><div class="line">  logger.debug(<span class="string">"[LOG] On: "</span> + Kue.getCertainJobAddress(event, <span class="keyword">this</span>));</div><div class="line">  eventBus.consumer(Kue.getCertainJobAddress(event, <span class="keyword">this</span>), handler);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">emit</span><span class="params">(String event, Object msg)</span> </span>&#123;</div><div class="line">  logger.debug(<span class="string">"[LOG] Emit: "</span> + Kue.getCertainJobAddress(event, <span class="keyword">this</span>));</div><div class="line">  eventBus.send(Kue.getCertainJobAddress(event, <span class="keyword">this</span>), msg);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在后面的代码中，我们将频繁使用这两个辅助函数。</p>
<h2 id="Redis中的存储形式">Redis中的存储形式</h2><p>在我们探索相关的逻辑函数之前，我们先来描述一下Vert.x Kue的数据在Redis中是以什么样的形式存储的：</p>
<ul>
<li>所有的key都在<code>vertx_kue</code>命名空间下(以<code>vertx_kue:</code>作为前缀)</li>
<li><code>vertx:kue:job:{id}</code>: 存储任务实体的map</li>
<li><code>vertx:kue:ids</code>: 计数器，指示当前最大的任务ID</li>
<li><code>vertx:kue:job:types</code>: 存储所有任务类型的列表</li>
<li><code>vertx:kue:{type}:jobs</code>: 指示所有等待状态下的某种类型任务的列表</li>
<li><code>vertx_kue:jobs</code>: 存储所有任务<code>zid</code>的有序集合</li>
<li><code>vertx_kue:job:{state}</code>: 存储所有指定状态的任务<code>zid</code>的有序集合</li>
<li><code>vertx_kue:jobs:{type}:{state}</code>: 存储所有指定状态和类型的任务<code>zid</code>的有序集合</li>
<li><code>vertx:kue:job:{id}:log</code>: 存储指定<code>id</code>的任务对应日志的列表</li>
</ul>
<p>OK，下面我们就来看看<code>Job</code>类中重要的逻辑函数。</p>
<h2 id="改变任务状态">改变任务状态</h2><p>我们之前提到过，Vert.x Kue中的任务一共有五种状态。所有的任务相关的操作都伴随着任务状态的变换，因此我们先来看一下<code>state</code>方法的实现，它用于改变任务的状态：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Job&gt; <span class="title">state</span><span class="params">(JobState newState)</span> </span>&#123;</div><div class="line">  Future&lt;Job&gt; future = Future.future();</div><div class="line">  RedisClient client = RedisHelper.client(vertx, <span class="keyword">new</span> JsonObject()); <span class="comment">// use a new client to keep transaction</span></div><div class="line">  JobState oldState = <span class="keyword">this</span>.state;</div><div class="line">  client.transaction().multi(r0 -&gt; &#123; <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">if</span> (r0.succeeded()) &#123;</div><div class="line">      <span class="keyword">if</span> (oldState != <span class="keyword">null</span> &amp;&amp; !oldState.equals(newState)) &#123; <span class="comment">// (2)</span></div><div class="line">        client.transaction().zrem(RedisHelper.getStateKey(oldState), <span class="keyword">this</span>.zid, _failure())</div><div class="line">          .zrem(RedisHelper.getKey(<span class="string">"jobs:"</span> + <span class="keyword">this</span>.type + <span class="string">":"</span> + oldState.name()), <span class="keyword">this</span>.zid, _failure());</div><div class="line">      &#125;</div><div class="line">      client.transaction().hset(RedisHelper.getKey(<span class="string">"job:"</span> + <span class="keyword">this</span>.id), <span class="string">"state"</span>, newState.name(), _failure()) <span class="comment">// (3)</span></div><div class="line">        .zadd(RedisHelper.getKey(<span class="string">"jobs:"</span> + newState.name()), <span class="keyword">this</span>.priority.getValue(), <span class="keyword">this</span>.zid, _failure())</div><div class="line">        .zadd(RedisHelper.getKey(<span class="string">"jobs:"</span> + <span class="keyword">this</span>.type + <span class="string">":"</span> + newState.name()), <span class="keyword">this</span>.priority.getValue(), <span class="keyword">this</span>.zid, _failure());</div><div class="line"></div><div class="line">      <span class="keyword">switch</span> (newState) &#123; <span class="comment">// dispatch different state</span></div><div class="line">        <span class="keyword">case</span> ACTIVE: <span class="comment">// (4)</span></div><div class="line">          client.transaction().zadd(RedisHelper.getKey(<span class="string">"jobs:"</span> + newState.name()),</div><div class="line">            <span class="keyword">this</span>.priority.getValue() &lt; <span class="number">0</span> ? <span class="keyword">this</span>.priority.getValue() : -<span class="keyword">this</span>.priority.getValue(),</div><div class="line">            <span class="keyword">this</span>.zid, _failure());</div><div class="line">          <span class="keyword">break</span>;</div><div class="line">        <span class="keyword">case</span> DELAYED: <span class="comment">// (5)</span></div><div class="line">          client.transaction().zadd(RedisHelper.getKey(<span class="string">"jobs:"</span> + newState.name()),</div><div class="line">            <span class="keyword">this</span>.promote_at, <span class="keyword">this</span>.zid, _failure());</div><div class="line">          <span class="keyword">break</span>;</div><div class="line">        <span class="keyword">case</span> INACTIVE: <span class="comment">// (6)</span></div><div class="line">          client.transaction().lpush(RedisHelper.getKey(<span class="keyword">this</span>.type + <span class="string">":jobs"</span>), <span class="string">"1"</span>, _failure());</div><div class="line">          <span class="keyword">break</span>;</div><div class="line">        <span class="keyword">default</span>:</div><div class="line">      &#125;</div><div class="line"></div><div class="line">      <span class="keyword">this</span>.state = newState;</div><div class="line"></div><div class="line">      client.transaction().exec(r -&gt; &#123; <span class="comment">// (7)</span></div><div class="line">        <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">          future.complete(<span class="keyword">this</span>);</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">          future.fail(r.cause());</div><div class="line">        &#125;</div><div class="line">      &#125;);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      future.fail(r0.cause());</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line"></div><div class="line">  <span class="keyword">return</span> future.compose(Job::updateNow);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们先创建了一个<code>Future</code>对象。然后我们调用了 <code>client.transaction().multi(handler)</code> 函数开始一次Redis事务 (1)。在Vert.x 3.3.2中，所有的Redis事务操作都移至<code>RedisTransaction</code>类中，所以我们需要先调用<code>client.transaction()</code>方法去获取一个事务实例，然后调用<code>multi</code>代表事务块的开始。</p>
<p>在<code>multi</code>函数传入的<code>Handler</code>中，我们先判定当前的任务状态。如果当前任务状态不为空并且不等于新的任务状态，我们就将Redis中存储的旧的状态信息移除 (2)。为了方便起见，我们提供了一个<code>RedisHelper</code>辅助类，里面提供了一些生成特定地址以及编码解码<code>zid</code>的方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.kue.util;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.kue.queue.JobState;</div><div class="line"><span class="keyword">import</span> io.vertx.core.Vertx;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.JsonObject;</div><div class="line"><span class="keyword">import</span> io.vertx.redis.RedisClient;</div><div class="line"><span class="keyword">import</span> io.vertx.redis.RedisOptions;</div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="class"><span class="keyword">class</span> <span class="title">RedisHelper</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String VERTX_KUE_REDIS_PREFIX = <span class="string">"vertx_kue"</span>;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="title">RedisHelper</span><span class="params">()</span> </span>&#123;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> RedisClient <span class="title">client</span><span class="params">(Vertx vertx, JsonObject config)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> RedisClient.create(vertx, options(config));</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> RedisOptions <span class="title">options</span><span class="params">(JsonObject config)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">new</span> RedisOptions()</div><div class="line">      .setHost(config.getString(<span class="string">"redis.host"</span>, <span class="string">"127.0.0.1"</span>))</div><div class="line">      .setPort(config.getInteger(<span class="string">"redis.port"</span>, <span class="number">6379</span>));</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> String <span class="title">getKey</span><span class="params">(String key)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> VERTX_KUE_REDIS_PREFIX + <span class="string">":"</span> + key;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> String <span class="title">getStateKey</span><span class="params">(JobState state)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> VERTX_KUE_REDIS_PREFIX + <span class="string">":jobs:"</span> + state.name();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> String <span class="title">createFIFO</span><span class="params">(<span class="keyword">long</span> id)</span> </span>&#123;</div><div class="line">    String idLen = <span class="string">""</span> + (<span class="string">""</span> + id).length();</div><div class="line">    <span class="keyword">int</span> len = <span class="number">2</span> - idLen.length();</div><div class="line">    <span class="keyword">while</span> (len-- &gt; <span class="number">0</span>)</div><div class="line">      idLen = <span class="string">"0"</span> + idLen;</div><div class="line">    <span class="keyword">return</span> idLen + <span class="string">"|"</span> + id;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> String <span class="title">stripFIFO</span><span class="params">(String zid)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> zid.substring(zid.indexOf(<span class="string">'|'</span>) + <span class="number">1</span>);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">long</span> <span class="title">numStripFIFO</span><span class="params">(String zid)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> Long.parseLong(zid.substring(zid.indexOf(<span class="string">'|'</span>) + <span class="number">1</span>));</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>所有的key都必须在<code>vertx_kue</code>命名空间下，因此我们封装了一个<code>getKey</code>方法。我们还实现了<code>createFIFO</code>和<code>stripFIFO</code>方法用于生成<code>zid</code>以及解码<code>zid</code>。<code>zid</code>的格式使用了Automattic/Kue中的格式。</p>
<p>回到<code>state</code>方法来。我们使用<code>zrem(String key, String member, Handler&lt;AsyncResult&lt;String&gt;&gt; handler)</code>方法将特定的数据从有序集合中移除。两个key分别是<code>vertx_kue:job:{state}</code> 以及 <code>vertx_kue:jobs:{type}:{state}</code>；<code>member</code>对应着任务的<code>zid</code>。</p>
<p>接下来我们使用<code>hset</code>方法来变更新的状态 (3)，然后用<code>zadd</code>方法往<code>vertx_kue:job:{state}</code> 和 <code>vertx_kue:jobs:{type}:{state}</code>两个有序集合中添加此任务的<code>zid</code>，同时传递一个<strong>权重</strong>(score)。这个非常重要，我们就是通过这个实现<strong>优先级</strong>队列的。我们直接使用<code>priority</code>对应的值作为<code>score</code>。这样，当我们需要从Redis中获取任务的时候，我们就可以通过<code>zpop</code>方法获取优先级最高的任务。我们会在后面详细讲述。</p>
<p>不同的新状态需要不同的操作。对于<code>ACTIVE</code>状态，我们通过<code>zadd</code>命令将<code>zid</code>添加至<code>vertx_kue:jobs:ACTIVE</code>有序集合中并赋予优先级权值 (4)。对于<code>DELAYED</code>状态，我们通过<code>zadd</code>命令将<code>zid</code>添加至<code>vertx_kue:jobs:DELAYED</code>有序集合中并赋予提升时间(<code>promote_at</code>)权值 (5)。对于<code>INACTIVE</code>状态，我们向<code>vertx:kue:{type}:jobs</code>列表中添加一个元素 (6)。这些操作都是在Redis事务块内完成的。最后我们通过<code>exec</code>方法一并执行这些事务操作 (7)。如果执行成功，我们给<code>future</code>赋值（当前任务）。最后我们返回<code>future</code>并且与<code>updateNow</code>方法相组合。</p>
<p><code>updateNow</code>方法非常简单，就是把<code>updated_at</code>的值设为当前时间，然后存到Redis中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="function">Future&lt;Job&gt; <span class="title">updateNow</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.updated_at = System.currentTimeMillis();</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>.set(<span class="string">"updated_at"</span>, String.valueOf(updated_at));</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<h2 id="保存任务">保存任务</h2><p>这里我们来看一下整个<code>Job</code>类中最重要的方法之一 - <code>save</code>方法，它的作用是保存任务至Redis中。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Job&gt; <span class="title">save</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="comment">// check</span></div><div class="line">  Objects.requireNonNull(<span class="keyword">this</span>.type, <span class="string">"Job type cannot be null"</span>); <span class="comment">// (1)</span></div><div class="line"></div><div class="line">  <span class="keyword">if</span> (<span class="keyword">this</span>.id &gt; <span class="number">0</span>)</div><div class="line">    <span class="keyword">return</span> update(); <span class="comment">// (2)</span></div><div class="line"></div><div class="line">  Future&lt;Job&gt; future = Future.future();</div><div class="line"></div><div class="line">  <span class="comment">// 生成id</span></div><div class="line">  client.incr(RedisHelper.getKey(<span class="string">"ids"</span>), res -&gt; &#123; <span class="comment">// (3)</span></div><div class="line">    <span class="keyword">if</span> (res.succeeded()) &#123;</div><div class="line">      <span class="keyword">this</span>.id = res.result();</div><div class="line">      <span class="keyword">this</span>.zid = RedisHelper.createFIFO(id); <span class="comment">// (4)</span></div><div class="line">      String key = RedisHelper.getKey(<span class="string">"job:"</span> + <span class="keyword">this</span>.id);</div><div class="line"></div><div class="line">      <span class="keyword">if</span> (<span class="keyword">this</span>.delay &gt; <span class="number">0</span>) &#123;</div><div class="line">        <span class="keyword">this</span>.state = JobState.DELAYED;</div><div class="line">      &#125;</div><div class="line"></div><div class="line">      client.sadd(RedisHelper.getKey(<span class="string">"job:types"</span>), <span class="keyword">this</span>.type, _failure()); <span class="comment">// (5)</span></div><div class="line">       <span class="keyword">this</span>.created_at = System.currentTimeMillis();</div><div class="line">       <span class="keyword">this</span>.promote_at = <span class="keyword">this</span>.created_at + <span class="keyword">this</span>.delay;</div><div class="line">       <span class="comment">// 保存任务</span></div><div class="line">       client.hmset(key, <span class="keyword">this</span>.toJson(), _completer(future, <span class="keyword">this</span>)); <span class="comment">// (6)</span></div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      future.fail(res.cause());</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line"></div><div class="line">  <span class="keyword">return</span> future.compose(Job::update); <span class="comment">// (7)</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先，任务类型不能为空所以我们要检查<code>type</code>是否为空 (1)。接着，如果当前任务的id大于0，则代表此任务已经存储过（因为id是存储时分配），此时只需执行更新操作(<code>update</code>)即可 (2)。然后我们创建一个<code>Future</code>对象，然后使用<code>incr</code>方法从<code>vertx_kue:ids</code>字段获取一个新的<code>id</code> (3)。同时我们使用<code>RedisHelper.createFIFO(id)</code>方法来生成新的<code>zid</code> (4)。接着我们来判断任务延时是否大于0，若大于0则将当前任务状态设置为<code>DELAYED</code>。然后我们通过<code>sadd</code>方法将当前任务类型添加至<code>vertx:kue:job:types</code>列表中 (5) 并且保存任务创建时间(<code>created_at</code>)以及任务提升时间(<code>promote_at</code>)。经过这一系列的操作后，所有的属性都已准备好，所以我们可以利用<code>hmset</code>方法将此任务实体存储至<code>vertx:kue:job:{id}</code>哈希表中 (6)。如果存储操作成功，那么将当前任务实体赋给<code>future</code>，否则记录错误。最后我们返回此<code>future</code>并且将其与<code>update</code>方法进行组合。</p>
<p><code>update</code>方法进行一些更新操作，它的逻辑比较简单：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="function">Future&lt;Job&gt; <span class="title">update</span><span class="params">()</span> </span>&#123;</div><div class="line">  Future&lt;Job&gt; future = Future.future();</div><div class="line">  <span class="keyword">this</span>.updated_at = System.currentTimeMillis();</div><div class="line"></div><div class="line">  client.transaction().multi(_failure())</div><div class="line">    .hset(RedisHelper.getKey(<span class="string">"job:"</span> + <span class="keyword">this</span>.id), <span class="string">"updated_at"</span>, String.valueOf(<span class="keyword">this</span>.updated_at), _failure())</div><div class="line">    .zadd(RedisHelper.getKey(<span class="string">"jobs"</span>), <span class="keyword">this</span>.priority.getValue(), <span class="keyword">this</span>.zid, _failure())</div><div class="line">    .exec(_completer(future, <span class="keyword">this</span>));</div><div class="line"></div><div class="line">  <span class="keyword">return</span> future.compose(r -&gt;</div><div class="line">    <span class="keyword">this</span>.state(<span class="keyword">this</span>.state));</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到<code>update</code>方法只做了三件微小的工作：存储任务更新时间、存储<code>zid</code>以及更改当前任务状态（组合<code>state</code>方法）。</p>
<p>最后总结一下将一个任务存储到Redis中经过的步骤：<code>save -&gt; update -&gt; state</code> :-)</p>
<h2 id="移除任务">移除任务</h2><p>移除任务非常简单，借助<code>zrem</code>和<code>del</code>方法即可。我们来看一下其实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Void&gt; <span class="title">remove</span><span class="params">()</span> </span>&#123;</div><div class="line">  Future&lt;Void&gt; future = Future.future();</div><div class="line">  client.transaction().multi(_failure())</div><div class="line">    .zrem(RedisHelper.getKey(<span class="string">"jobs:"</span> + <span class="keyword">this</span>.stateName()), <span class="keyword">this</span>.zid, _failure())</div><div class="line">    .zrem(RedisHelper.getKey(<span class="string">"jobs:"</span> + <span class="keyword">this</span>.type + <span class="string">":"</span> + <span class="keyword">this</span>.stateName()), <span class="keyword">this</span>.zid, _failure())</div><div class="line">    .zrem(RedisHelper.getKey(<span class="string">"jobs"</span>), <span class="keyword">this</span>.zid, _failure())</div><div class="line">    .del(RedisHelper.getKey(<span class="string">"job:"</span> + <span class="keyword">this</span>.id + <span class="string">":log"</span>), _failure())</div><div class="line">    .del(RedisHelper.getKey(<span class="string">"job:"</span> + <span class="keyword">this</span>.id), _failure())</div><div class="line">    .exec(r -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">        <span class="keyword">this</span>.emit(<span class="string">"remove"</span>, <span class="keyword">new</span> JsonObject().put(<span class="string">"id"</span>, <span class="keyword">this</span>.id));</div><div class="line">        future.complete();</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        future.fail(r.cause());</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">  <span class="keyword">return</span> future;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>注意到成功移除任务时，我们会向Event Bus上的特定地址发送<code>remove</code>任务事件。此事件包含着被移除任务的<code>id</code>。</p>
<h2 id="监听任务事件">监听任务事件</h2><p>我们可以通过几种 <code>onXXX</code> 方法来监听任务事件：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">onComplete</span><span class="params">(Handler&lt;Job&gt; completeHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.on(<span class="string">"complete"</span>, message -&gt; &#123;</div><div class="line">    completeHandler.handle(<span class="keyword">new</span> Job((JsonObject) message.body()));</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">onFailure</span><span class="params">(Handler&lt;JsonObject&gt; failureHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.on(<span class="string">"failed"</span>, message -&gt; &#123;</div><div class="line">    failureHandler.handle((JsonObject) message.body());</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">onFailureAttempt</span><span class="params">(Handler&lt;JsonObject&gt; failureHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.on(<span class="string">"failed_attempt"</span>, message -&gt; &#123;</div><div class="line">    failureHandler.handle((JsonObject) message.body());</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">onPromotion</span><span class="params">(Handler&lt;Job&gt; handler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.on(<span class="string">"promotion"</span>, message -&gt; &#123;</div><div class="line">    handler.handle(<span class="keyword">new</span> Job((JsonObject) message.body()));</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">onStart</span><span class="params">(Handler&lt;Job&gt; handler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.on(<span class="string">"start"</span>, message -&gt; &#123;</div><div class="line">    handler.handle(<span class="keyword">new</span> Job((JsonObject) message.body()));</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">onRemove</span><span class="params">(Handler&lt;JsonObject&gt; removeHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.on(<span class="string">"start"</span>, message -&gt; &#123;</div><div class="line">    removeHandler.handle((JsonObject) message.body());</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">onProgress</span><span class="params">(Handler&lt;Integer&gt; progressHandler)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.on(<span class="string">"progress"</span>, message -&gt; &#123;</div><div class="line">    progressHandler.handle((Integer) message.body());</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>注意到不同的事件，对应接收的数据类型也有差异。我们来说明一下：</p>
<ul>
<li><code>onComplete</code>、<code>onPromotion</code> 以及 <code>onStart</code>: 发送的数据是对应的<code>Job</code>对象</li>
<li><code>onFailure</code> and <code>onFailureAttempt</code>: 发送的数据是<code>JsonObject</code>类型的，其格式类似于：</li>
</ul>
<figure class="highlight json"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line">&#123;</div><div class="line">    <span class="attr">"job"</span>: &#123;&#125;,</div><div class="line">    <span class="attr">"extra"</span>: &#123;</div><div class="line">        <span class="attr">"message"</span>: <span class="string">"some_error"</span></div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<ul>
<li><code>onProgress</code>: 发送的数据是当前任务进度</li>
<li><code>onRemove</code>: 发送的数据是<code>JsonObject</code>类型的，其中<code>id</code>代表被移除任务的编号</li>
</ul>
<h2 id="更新任务进度">更新任务进度</h2><p>我们可以通过<code>progress</code>方法来更新任务进度。看一下其实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Job&gt; <span class="title">progress</span><span class="params">(<span class="keyword">int</span> complete, <span class="keyword">int</span> total)</span> </span>&#123;</div><div class="line">  <span class="keyword">int</span> n = Math.min(<span class="number">100</span>, complete * <span class="number">100</span> / total); <span class="comment">// (1)</span></div><div class="line">  <span class="keyword">this</span>.emit(<span class="string">"progress"</span>, n); <span class="comment">// (2)</span></div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>.setProgress(n) <span class="comment">// (3)</span></div><div class="line">    .set(<span class="string">"progress"</span>, String.valueOf(n))</div><div class="line">    .compose(Job::updateNow);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p><code>progress</code>方法接受两个参数：第一个是当前完成的进度值，第二个是完成状态需要的进度值。我们首先计算出当前的进度 (1)，然后向特定地址发送<code>progress</code>事件 (2)。最后我们将进度存储至Redis中并更新时间，返回<code>Future</code> (3)。</p>
<h2 id="任务失败以及重试机制">任务失败以及重试机制</h2><p>当一个任务处理失败时，如果它有剩余的重试次数，Vert.x Kue会自动调用<code>failAttempt</code>方法进行重试。我们来看一下<code>failAttempt</code>方法的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line"><span class="function">Future&lt;Job&gt; <span class="title">failedAttempt</span><span class="params">(Throwable err)</span> </span>&#123;</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>.error(err)</div><div class="line">    .compose(Job::failed)</div><div class="line">    .compose(Job::attemptInternal);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>(⊙o⊙)非常简短吧～实际上，<code>failAttempt</code>方法是三个异步方法的组合：<code>error</code>、<code>failed</code>以及<code>attemptInternal</code>。当一个任务需要进行重试的时候，我们首先向Event Bus发布 <code>error</code> 队列事件并且在Redis中记录日志，然后将当前的任务状态置为<code>FAILED</code>，最后重新处理此任务。</p>
<p>我们先来看一下<code>error</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Job&gt; <span class="title">error</span><span class="params">(Throwable ex)</span> </span>&#123;</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>.emitError(ex)</div><div class="line">    .set(<span class="string">"error"</span>, ex.getMessage())</div><div class="line">    .compose(j -&gt; j.log(<span class="string">"error | "</span> + ex.getMessage()));</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>它的逻辑很简单：首先我们向Event Bus发布 <strong>错误</strong> 事件，然后记录错误日志即可。这里我们封装了一个发布错误的函数<code>emitError</code>：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function"><span class="keyword">public</span> Job <span class="title">emitError</span><span class="params">(Throwable ex)</span> </span>&#123;</div><div class="line">  JsonObject errorMessage = <span class="keyword">new</span> JsonObject().put(<span class="string">"id"</span>, <span class="keyword">this</span>.id)</div><div class="line">    .put(<span class="string">"message"</span>, ex.getMessage());</div><div class="line">  eventBus.publish(Kue.workerAddress(<span class="string">"error"</span>), errorMessage);</div><div class="line">  eventBus.send(Kue.getCertainJobAddress(<span class="string">"error"</span>, <span class="keyword">this</span>), errorMessage);</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>其中发送的错误信息格式类似于下面的样子：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">&#123;</div><div class="line">    <span class="attr">"id"</span>: <span class="number">2052</span>,</div><div class="line">    <span class="attr">"message"</span>: <span class="string">"some error"</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>接下来我们再来看一下<code>failed</code>方法的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Job&gt; <span class="title">failed</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.failed_at = System.currentTimeMillis();</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>.updateNow()</div><div class="line">    .compose(j -&gt; j.set(<span class="string">"failed_at"</span>, String.valueOf(j.failed_at)))</div><div class="line">    .compose(j -&gt; j.state(JobState.FAILED));</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>非常简单，首先我们更新任务的更新时间和失败时间，然后通过<code>state</code>方法将当前任务状态置为<code>FAILED</code>即可。</p>
<p>任务重试的核心逻辑在<code>attemptInternal</code>方法中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> Future&lt;Job&gt; <span class="title">attemptInternal</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">int</span> remaining = <span class="keyword">this</span>.max_attempts - <span class="keyword">this</span>.attempts; <span class="comment">// (1)</span></div><div class="line">  <span class="keyword">if</span> (remaining &gt; <span class="number">0</span>) &#123; <span class="comment">// 还有重试次数</span></div><div class="line">    <span class="keyword">return</span> <span class="keyword">this</span>.attemptAdd() <span class="comment">// (2)</span></div><div class="line">      .compose(Job::reattempt) <span class="comment">// (3)</span></div><div class="line">      .setHandler(r -&gt; &#123;</div><div class="line">        <span class="keyword">if</span> (r.failed()) &#123;</div><div class="line">          <span class="keyword">this</span>.emitError(r.cause()); <span class="comment">// (4)</span></div><div class="line">        &#125;</div><div class="line">      &#125;);</div><div class="line">  &#125; <span class="keyword">else</span> <span class="keyword">if</span> (remaining == <span class="number">0</span>) &#123; <span class="comment">// (5)</span></div><div class="line">    <span class="keyword">return</span> Future.failedFuture(<span class="string">"No more attempts"</span>);</div><div class="line">  &#125; <span class="keyword">else</span> &#123; <span class="comment">// (6)</span></div><div class="line">    <span class="keyword">return</span> Future.failedFuture(<span class="keyword">new</span> IllegalStateException(<span class="string">"Attempts Exceeded"</span>));</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在我们的<code>Job</code>数据对象中，我们存储了最大重试次数<code>max_attempts</code>以及已经重试的次数<code>attempts</code>，所以我们首先根据这两个数据计算剩余的重试次数<code>remaining</code> (1)。如果还有剩余次数的话，我们就先调用<code>attemptAdd</code>方法增加一次已重试次数并 (2)，然后我们调用<code>reattempt</code>方法执行真正的任务重试逻辑 (3)。最后返回这两个异步方法组合的<code>Future</code>。如果其中一个过程出现错误，我们就发布<code>error</code>事件 (4)。如果没有剩余次数了或者超出剩余次数了，我们直接返回错误。</p>
<p>在我们解析<code>reattempt</code>方法之前，我们先来回顾一下Vert.x Kue中的任务失败恢复机制。Vert.x Kue支持延时重试机制(retry backoff)，并且支持不同的策略（如 <strong>fixed</strong> 以及 <strong>exponential</strong>）。之前我们提到<code>Job</code>类中有一个<code>backoff</code>成员变量，它用于配置延时重试的策略。它的格式类似于这样：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">&#123;</div><div class="line">    <span class="attr">"type"</span>: <span class="string">"fixed"</span>,</div><div class="line">    <span class="attr">"delay"</span>: <span class="number">5000</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>延时重试机制的实现在<code>getBackoffImpl</code>方法中，它返回一个<code>Function&lt;Integer, Long&gt;</code>对象，代表一个接受<code>Integer</code>类型（即<code>attempts</code>），返回<code>Long</code>类型（代表计算出的延时值）的函数：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> Function&lt;Integer, Long&gt; <span class="title">getBackoffImpl</span><span class="params">()</span> </span>&#123;</div><div class="line">  String type = <span class="keyword">this</span>.backoff.getString(<span class="string">"type"</span>, <span class="string">"fixed"</span>); <span class="comment">// (1)</span></div><div class="line">  <span class="keyword">long</span> _delay = <span class="keyword">this</span>.backoff.getLong(<span class="string">"delay"</span>, <span class="keyword">this</span>.delay); <span class="comment">// (2)</span></div><div class="line">  <span class="keyword">switch</span> (type) &#123;</div><div class="line">    <span class="keyword">case</span> <span class="string">"exponential"</span>: <span class="comment">// (3)</span></div><div class="line">      <span class="keyword">return</span> attempts -&gt; Math.round(_delay * <span class="number">0.5</span> * (Math.pow(<span class="number">2</span>, attempts) - <span class="number">1</span>));</div><div class="line">    <span class="keyword">case</span> <span class="string">"fixed"</span>:</div><div class="line">    <span class="keyword">default</span>: <span class="comment">// (4)</span></div><div class="line">      <span class="keyword">return</span> attempts -&gt; _delay;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们从<code>backoff</code>配置中获取延迟重试策略。目前Vert.x Kue支持两种策略：<code>fixed</code> 和 <code>exponential</code>。前者采用固定延迟时间，而后者采用指数增长型延迟时间。默认情况下Vert.x Kue会采用<code>fixed</code>策略 (1)。接下来我们从<code>backoff</code>配置中获取延迟时间，如果配置中没有指定，那么就使用任务对象中的延迟时间<code>delay</code> (2)。接下来就是根据具体的策略进行计算了。对于指数型延迟，我们计算<code>[delay * 0.5 * 2^attempts]</code>作为延迟时间 (3)；对于固定型延迟策略，我们直接使用获取到的延迟时间 (4)。</p>
<p>好啦，现在回到“真正的重试”方法 —— <code>reattempt</code>方法来：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> Future&lt;Job&gt; <span class="title">reattempt</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (<span class="keyword">this</span>.backoff != <span class="keyword">null</span>) &#123;</div><div class="line">    <span class="keyword">long</span> delay = <span class="keyword">this</span>.getBackoffImpl().apply(attempts); <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">return</span> <span class="keyword">this</span>.setDelay(delay)</div><div class="line">      .setPromote_at(System.currentTimeMillis() + delay)</div><div class="line">      .update() <span class="comment">// (2)</span></div><div class="line">      .compose(Job::delayed); <span class="comment">// (3)</span></div><div class="line">  &#125; <span class="keyword">else</span> &#123;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">this</span>.inactive(); <span class="comment">// (4)</span></div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们先检查<code>backoff</code>配置是否存在，若存在则计算出对应的延时时间 (1) 并且设定<code>delay</code>和<code>promote_at</code>属性的值然后保存至Redis中 (2)。接着我们通过<code>delayed</code>方法将任务的状态设为延时(<code>DELAYED</code>) (3)。如果延时重试配置不存在，我们就通过<code>inactive</code>方法直接将此任务置入工作队列中 (4)。</p>
<p>这就是整个任务重试功能的实现，也不是很复杂蛤？观察上面的代码，我们可以发现<code>Future</code>组合无处不在。这种响应式的组合非常方便。想一想如果我们用回调的异步方式来写代码的话，我们很容易陷入回调地狱中(⊙o⊙)。。。几个回调嵌套起来总显得不是那么优美和简洁，而用响应式的、可组合的<code>Future</code>就可以有效地避免这个问题。</p>
<p>不错！到现在为止我们已经探索完<code>Job</code>类的源码了～下面我们来看一下<code>JobService</code>类。</p>
<h1 id="Event_Bus_服务_-_JobService">Event Bus 服务 - JobService</h1><p>在本章节中我们来探索一下<code>JobService</code>接口及其实现 —— 它包含着各种普通的操作和统计<code>Job</code>的逻辑。</p>
<h2 id="异步RPC">异步RPC</h2><p>我们的<code>JobService</code>是一个通用逻辑接口，因此我们希望应用中的每一个组件都能访问此服务，即进行RPC。在Vert.x中，我们可以将服务注册至Event Bus上，然后其它组件就可以通过Event Bus来远程调用注册的服务了。</p>
<p>传统的RPC有一个缺点：消费者需要阻塞等待生产者的回应。你可能想说：这是一种阻塞模型，和Vert.x推崇的异步开发模式不相符。没错！而且，传统的RPC不是真正<strong>面向失败设计</strong>的。</p>
<p>还好，Vert.x提供了一种高效的、响应式的RPC —— 异步RPC。我们不需要等待生产者的回应，而只需要传递一个<code>Handler&lt;AsyncResult&lt;R&gt;&gt;</code>参数给异步方法。这样当收到生产者结果时，对应的<code>Handler</code>就会被调用，非常方便，这与Vert.x的异步开发模式相符。并且，<code>AsyncResult</code>也是面向失败设计的。</p>
<p>所以讲到这里，你可能想问：到底怎么在Event Bus上注册服务呢？我们是不是需要写一大堆的逻辑去包装和发送信息，然后在另一端解码信息并进行调用呢？不，这太麻烦了！有了Vert.x 服务代理，我们不需要这么做！Vert.x提供了一个组件 <strong>Vert.x Service Proxy</strong> 来自动生成服务代理。有了它的帮助，我们就只需要按照规范设计我们的异步服务接口，然后用<code>@ProxyGen</code>注解修饰即可。</p>
<p>[NOTE <code>@ProxyGen</code>注解的限制 | <code>@ProxyGen</code>注解的使用有诸多限制。比如，所有的异步方法都必须是基于回调的，也就是说每个方法都要接受一个<code>Handler&lt;AsyncResult&lt;R&gt;&gt;</code>类型的参数。并且，类型<code>R</code>也是有限制的 —— 只允许基本类型以及数据对象类型。详情请参考<a href="http://vertx.io/docs/vertx-service-proxy/" target="_blank" rel="external">官方文档</a>。 ]</p>
<h2 id="异步服务接口">异步服务接口</h2><p>我们来看一下<code>JobService</code>的源码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div><div class="line">68</div><div class="line">69</div><div class="line">70</div><div class="line">71</div><div class="line">72</div><div class="line">73</div><div class="line">74</div><div class="line">75</div><div class="line">76</div><div class="line">77</div><div class="line">78</div><div class="line">79</div><div class="line">80</div><div class="line">81</div><div class="line">82</div><div class="line">83</div><div class="line">84</div><div class="line">85</div><div class="line">86</div><div class="line">87</div><div class="line">88</div><div class="line">89</div><div class="line">90</div><div class="line">91</div><div class="line">92</div><div class="line">93</div><div class="line">94</div><div class="line">95</div><div class="line">96</div><div class="line">97</div><div class="line">98</div><div class="line">99</div><div class="line">100</div><div class="line">101</div><div class="line">102</div><div class="line">103</div><div class="line">104</div><div class="line">105</div><div class="line">106</div><div class="line">107</div><div class="line">108</div><div class="line">109</div><div class="line">110</div><div class="line">111</div><div class="line">112</div><div class="line">113</div><div class="line">114</div><div class="line">115</div><div class="line">116</div><div class="line">117</div><div class="line">118</div><div class="line">119</div><div class="line">120</div><div class="line">121</div><div class="line">122</div><div class="line">123</div><div class="line">124</div><div class="line">125</div><div class="line">126</div><div class="line">127</div><div class="line">128</div><div class="line">129</div><div class="line">130</div><div class="line">131</div><div class="line">132</div><div class="line">133</div><div class="line">134</div><div class="line">135</div><div class="line">136</div><div class="line">137</div><div class="line">138</div><div class="line">139</div><div class="line">140</div><div class="line">141</div><div class="line">142</div><div class="line">143</div><div class="line">144</div><div class="line">145</div><div class="line">146</div><div class="line">147</div><div class="line">148</div><div class="line">149</div><div class="line">150</div><div class="line">151</div><div class="line">152</div><div class="line">153</div><div class="line">154</div><div class="line">155</div><div class="line">156</div><div class="line">157</div><div class="line">158</div><div class="line">159</div><div class="line">160</div><div class="line">161</div><div class="line">162</div><div class="line">163</div><div class="line">164</div><div class="line">165</div><div class="line">166</div><div class="line">167</div><div class="line">168</div><div class="line">169</div><div class="line">170</div><div class="line">171</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@ProxyGen</span></div><div class="line"><span class="meta">@VertxGen</span></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">JobService</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">static</span> JobService <span class="title">create</span><span class="params">(Vertx vertx, JsonObject config)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">new</span> JobServiceImpl(vertx, config);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">static</span> JobService <span class="title">createProxy</span><span class="params">(Vertx vertx, String address)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> ProxyHelper.createProxy(JobService.class, vertx, address);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取任务，按照优先级顺序</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> id      job id</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">getJob</span><span class="params">(<span class="keyword">long</span> id, Handler&lt;AsyncResult&lt;Job&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 删除任务</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> id      job id</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">removeJob</span><span class="params">(<span class="keyword">long</span> id, Handler&lt;AsyncResult&lt;Void&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 判断任务是否存在</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> id      job id</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">existsJob</span><span class="params">(<span class="keyword">long</span> id, Handler&lt;AsyncResult&lt;Boolean&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取任务日志</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> id      job id</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">getJobLog</span><span class="params">(<span class="keyword">long</span> id, Handler&lt;AsyncResult&lt;JsonArray&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取某一范围内某个指定状态下的任务列表</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> state   expected job state</div><div class="line">   * <span class="doctag">@param</span> from    from</div><div class="line">   * <span class="doctag">@param</span> to      to</div><div class="line">   * <span class="doctag">@param</span> order   range order</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">jobRangeByState</span><span class="params">(String state, <span class="keyword">long</span> from, <span class="keyword">long</span> to, String order, Handler&lt;AsyncResult&lt;List&lt;Job&gt;&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取某一范围内某个指定状态和类型下的任务列表</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> type    expected job type</div><div class="line">   * <span class="doctag">@param</span> state   expected job state</div><div class="line">   * <span class="doctag">@param</span> from    from</div><div class="line">   * <span class="doctag">@param</span> to      to</div><div class="line">   * <span class="doctag">@param</span> order   range order</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">jobRangeByType</span><span class="params">(String type, String state, <span class="keyword">long</span> from, <span class="keyword">long</span> to, String order, Handler&lt;AsyncResult&lt;List&lt;Job&gt;&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取某一范围内的任务列表（按照顺序或倒序）</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> from    from</div><div class="line">   * <span class="doctag">@param</span> to      to</div><div class="line">   * <span class="doctag">@param</span> order   range order</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">jobRange</span><span class="params">(<span class="keyword">long</span> from, <span class="keyword">long</span> to, String order, Handler&lt;AsyncResult&lt;List&lt;Job&gt;&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">// 统计函数</span></div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取指定状态和类型下的任务的数量</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> type    job type</div><div class="line">   * <span class="doctag">@param</span> state   job state</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">cardByType</span><span class="params">(String type, JobState state, Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取某个状态下的任务的数量</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> state   job state</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">card</span><span class="params">(JobState state, Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取COMPLETE状态任务的数量</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> type    job type; if null, then return global metrics</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">completeCount</span><span class="params">(String type, Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取FAILED状态任务的数量</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> type job type; if null, then return global metrics</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">failedCount</span><span class="params">(String type, Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取INACTIVE状态任务的数量</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> type job type; if null, then return global metrics</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">inactiveCount</span><span class="params">(String type, Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取ACTIVE状态任务的数量</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> type job type; if null, then return global metrics</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">activeCount</span><span class="params">(String type, Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取DELAYED状态任务的数量</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> type job type; if null, then return global metrics</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">delayedCount</span><span class="params">(String type, Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取当前存在的所有任务类型</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">getAllTypes</span><span class="params">(Handler&lt;AsyncResult&lt;List&lt;String&gt;&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 获取指定状态下的所有任务的ID</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> state   job state</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">getIdsByState</span><span class="params">(JobState state, Handler&lt;AsyncResult&lt;List&lt;Long&gt;&gt;&gt; handler)</span></span>;</div><div class="line"></div><div class="line">  <span class="comment">/**</span></div><div class="line">   * 工作队列运行时间（ms）</div><div class="line">   *</div><div class="line">   * <span class="doctag">@param</span> handler async result handler</div><div class="line">   */</div><div class="line">  <span class="meta">@Fluent</span></div><div class="line">  <span class="function">JobService <span class="title">getWorkTime</span><span class="params">(Handler&lt;AsyncResult&lt;Long&gt;&gt; handler)</span></span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>可以看到我们还为<code>JobService</code>接口添加了<code>@VertxGen</code>注解，Vert.x Codegen可以处理此注解生成多种语言版本的服务。</p>
<p>在<code>JobService</code>接口中我们还定义了两个静态方法：<code>create</code>用于创建一个任务服务实例，<code>createProxy</code>用于创建一个服务代理。</p>
<p><code>JobService</code>接口中包含一些任务操作和统计的相关逻辑，每个方法的功能都已经在注释中阐述了，因此我们就直接来看它的实现吧～</p>
<h2 id="任务服务的实现">任务服务的实现</h2><p><code>JobService</code>接口的实现位于<code>JobServiceImpl</code>类中，代码非常长，因此这里就不贴代码了。。。大家可以对照<a href="https://github.com/sczyh30/vertx-blueprint-job-queue/blob/master/kue-core/src/main/java/io/vertx/blueprint/kue/service/impl/JobServiceImpl.java" target="_blank" rel="external">GitHub中的代码</a>读下面的内容。</p>
<ul>
<li><code>getJob</code>: 获取任务的方法非常简单。直接利用<code>hgetall</code>命令从Redis中取出对应的任务即可。</li>
<li><code>removeJob</code>: 我们可以将此方法看作是<code>getJob</code>和<code>Job#remove</code>两个方法的组合。</li>
<li><code>existsJob</code>: 使用<code>exists</code>命令判断对应<code>id</code>的任务是否存在。</li>
<li><code>getJobLog</code>: 使用<code>lrange</code>命令从<code>vertx_kue:job:{id}:log</code>列表中取出日志。</li>
<li><code>rangeGeneral</code>: 使用<code>zrange</code>命令获取一定范围内的任务，这是一个通用方法。</li>
</ul>
<p>[NOTE <code>zrange</code> 操作 | <code>zrange</code> 返回某一有序集合中某个特定范围内的元素。详情请见<a href="http://redis.io/commands/zrange" target="_blank" rel="external">ZRANGE - Redis</a>。 ]</p>
<p>以下三个方法复用了<code>rangeGeneral</code>方法：</p>
<ul>
<li><code>jobRangeByState</code>: 指定状态，对应的key为<code>vertx_kue:jobs:{state}</code>。</li>
<li><code>jobRangeByType</code>: 指定状态和类型，对应的key为<code>vertx_kue:jobs:{type}:{state}</code>。</li>
<li><code>jobRange</code>: 对应的key为<code>vertx_kue:jobs</code>。</li>
</ul>
<p>这两个通用方法用于任务数量的统计：</p>
<ul>
<li><code>cardByType</code>: 利用<code>zcard</code>命令获取某一指定状态和类型下任务的数量。</li>
<li><code>card</code>: 利用<code>zcard</code>命令获取某一指定状态下任务的数量。</li>
</ul>
<p>下面五个辅助统计方法复用了上面两个通用方法：</p>
<ul>
<li><code>completeCount</code></li>
<li><code>failedCount</code></li>
<li><code>delayedCount</code></li>
<li><code>inactiveCount</code></li>
<li><code>activeCount</code></li>
</ul>
<p>接着看：</p>
<ul>
<li><code>getAllTypes</code>: 利用<code>smembers</code>命令获取<code>vertx_kue:job:types</code>集合中存储的所有的任务类型。</li>
<li><code>getIdsByState</code>: 使用<code>zrange</code>获取某一指定状态下所有任务的ID。</li>
<li><code>getWorkTime</code>: 使用<code>get</code>命令从<code>vertx_kue:stats:work-time</code>中获取Vert.x Kue的工作时间。</li>
</ul>
<h2 id="注册任务服务">注册任务服务</h2><p>既然完成了<code>JobService</code>的实现，接下来我们来看一下如何利用Service Proxy将服务注册至Event Bus上。这里我们还需要一个<code>KueVerticle</code>来创建要注册的服务实例，并且将其注册至Event Bus上。</p>
<p>打开<code>io.vertx.blueprint.kue.queue.KueVerticle</code>类的源码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.kue.queue;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.kue.service.JobService;</div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.kue.util.RedisHelper;</div><div class="line"><span class="keyword">import</span> io.vertx.core.AbstractVerticle;</div><div class="line"><span class="keyword">import</span> io.vertx.core.Future;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.JsonObject;</div><div class="line"><span class="keyword">import</span> io.vertx.core.logging.Logger;</div><div class="line"><span class="keyword">import</span> io.vertx.core.logging.LoggerFactory;</div><div class="line"><span class="keyword">import</span> io.vertx.redis.RedisClient;</div><div class="line"><span class="keyword">import</span> io.vertx.serviceproxy.ProxyHelper;</div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">KueVerticle</span> <span class="keyword">extends</span> <span class="title">AbstractVerticle</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> Logger logger = LoggerFactory.getLogger(Job.class);</div><div class="line"></div><div class="line">  <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String EB_JOB_SERVICE_ADDRESS = <span class="string">"vertx.kue.service.job.internal"</span>; <span class="comment">// (1)</span></div><div class="line"></div><div class="line">  <span class="keyword">private</span> JsonObject config;</div><div class="line">  <span class="keyword">private</span> JobService jobService;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">start</span><span class="params">(Future&lt;Void&gt; future)</span> <span class="keyword">throws</span> Exception </span>&#123;</div><div class="line">    <span class="keyword">this</span>.config = config();</div><div class="line">    <span class="keyword">this</span>.jobService = JobService.create(vertx, config); <span class="comment">// (2)</span></div><div class="line">    <span class="comment">// create redis client</span></div><div class="line">    RedisClient redisClient = RedisHelper.client(vertx, config);</div><div class="line">    redisClient.ping(pr -&gt; &#123; <span class="comment">// (3) test connection</span></div><div class="line">      <span class="keyword">if</span> (pr.succeeded()) &#123;</div><div class="line">        logger.info(<span class="string">"Kue Verticle is running..."</span>);</div><div class="line"></div><div class="line">        <span class="comment">// (4) register job service</span></div><div class="line">        ProxyHelper.registerService(JobService.class, vertx, jobService, EB_JOB_SERVICE_ADDRESS);</div><div class="line"></div><div class="line">        future.complete();</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        logger.error(<span class="string">"oops!"</span>, pr.cause());</div><div class="line">        future.fail(pr.cause());</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们需要定义一个地址用于服务注册 (1)。在<code>start</code>方法中，我们创建了一个任务服务实例 (2)，然后通过<code>ping</code>命令测试Redis连接 (3)。如果连接正常，那么我们就可以通过<code>ProxyHelper</code>类中的<code>registerService</code>辅助方法来将服务实例注册至Event Bus上 (4)。</p>
<p>这样，一旦我们在集群模式下部署<code>KueVerticle</code>，服务就会被发布至Event Bus上，然后我们就可以在其他组件中去远程调用此服务了。很奇妙吧！</p>
<h1 id="Kue_-_工作队列">Kue - 工作队列</h1><p><code>Kue</code>类代表着工作队列。我们来看一下<code>Kue</code>类的实现。首先先看一下其构造函数：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="title">Kue</span><span class="params">(Vertx vertx, JsonObject config)</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.vertx = vertx;</div><div class="line">  <span class="keyword">this</span>.config = config;</div><div class="line">  <span class="keyword">this</span>.jobService = JobService.createProxy(vertx, EB_JOB_SERVICE_ADDRESS);</div><div class="line">  <span class="keyword">this</span>.client = RedisHelper.client(vertx, config);</div><div class="line">  Job.setVertx(vertx, RedisHelper.client(vertx, config)); <span class="comment">// init static vertx instance inner job</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里我们需要注意两点：第一点，我们通过<code>createProxy</code>方法来创建一个<code>JobService</code>的服务代理；第二点，之前提到过，我们需要在这里初始化<code>Job</code>类中的静态成员变量。</p>
<h2 id="基于Future的封装">基于Future的封装</h2><p>我们的<code>JobService</code>是基于回调的，这是服务代理组件所要求的。为了让Vert.x Kue更加响应式，使用起来更加方便，我们在<code>Kue</code>类中以基于Future的异步模式封装了<code>JobService</code>中的所有异步方法。这很简单，比如这个方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Fluent</span></div><div class="line"><span class="function">JobService <span class="title">getJob</span><span class="params">(<span class="keyword">long</span> id, Handler&lt;AsyncResult&lt;Job&gt;&gt; handler)</span></span>;</div></pre></td></tr></table></figure>
<p>可以这么封装：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> Future&lt;Optional&lt;Job&gt;&gt; getJob(<span class="keyword">long</span> id) &#123;</div><div class="line">  Future&lt;Optional&lt;Job&gt;&gt; future = Future.future();</div><div class="line">  jobService.getJob(id, r -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">      future.complete(Optional.ofNullable(r.result()));</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      future.fail(r.cause());</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> future;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>其实就是加一层<code>Future</code>。其它的封装过程也类似所以我们就不细说了。</p>
<p><img src="https://raw.githubusercontent.com/sczyh30/vertx-blueprint-job-queue/master/docs/images/kue_future_based_methods.png" alt=""></p>
<h2 id="process和processBlocking方法">process和processBlocking方法</h2><p><code>process</code>和<code>processBlocking</code>方法用于处理任务：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> Kue <span class="title">process</span><span class="params">(String type, <span class="keyword">int</span> n, Handler&lt;Job&gt; handler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (n &lt;= <span class="number">0</span>) &#123;</div><div class="line">    <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"The process times must be positive"</span>);</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">while</span> (n-- &gt; <span class="number">0</span>) &#123;</div><div class="line">    processInternal(type, handler, <span class="keyword">false</span>);</div><div class="line">  &#125;<span class="function">f</span></div><div class="line">  <span class="title">setupTimers</span><span class="params">()</span>;</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">public</span> Kue <span class="title">process</span><span class="params">(String type, Handler&lt;Job&gt; handler)</span> </span>&#123;</div><div class="line">  processInternal(type, handler, <span class="keyword">false</span>);</div><div class="line">  setupTimers();</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">public</span> Kue <span class="title">processBlocking</span><span class="params">(String type, <span class="keyword">int</span> n, Handler&lt;Job&gt; handler)</span> </span>&#123;</div><div class="line">  <span class="keyword">if</span> (n &lt;= <span class="number">0</span>) &#123;</div><div class="line">    <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"The process times must be positive"</span>);</div><div class="line">  &#125;</div><div class="line">  <span class="keyword">while</span> (n-- &gt; <span class="number">0</span>) &#123;</div><div class="line">    processInternal(type, handler, <span class="keyword">true</span>);</div><div class="line">  &#125;</div><div class="line">  setupTimers();</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>两个<code>process</code>方法都类似 —— 它们都是使用<strong>Event Loop</strong>线程处理任务的，其中第一个方法还可以指定同时处理任务数量的阈值。我们来回顾一下使用<strong>Event Loop</strong>线程的注意事项 —— 我们不能阻塞Event Loop线程。因此如果我们需要在处理任务时做一些耗时的操作，我们可以使用<code>processBlocking</code>方法。这几个方法的代码看起来都差不多，那么区别在哪呢？之前我们提到过，我们设计了一种Verticle - <code>KueWorker</code>，用于处理任务。因此对于<code>process</code>方法来说，<code>KueWorker</code>就是一种普通的Verticle；而对于<code>processBlocking</code>方法来说，<code>KueWorker</code>是一种<strong>Worker Verticle</strong>。这两种Verticle有什么不同呢？区别在于，Worker Verticle会使用<strong>Worker线程</strong>，因此即使我们执行一些耗时的操作，Event Loop线程也不会被阻塞。</p>
<p>创建及部署<code>KueWorker</code>的逻辑在<code>processInternal</code>方法中，这三个方法都使用了<code>processInternal</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">processInternal</span><span class="params">(String type, Handler&lt;Job&gt; handler, <span class="keyword">boolean</span> isWorker)</span> </span>&#123;</div><div class="line">  KueWorker worker = <span class="keyword">new</span> KueWorker(type, handler, <span class="keyword">this</span>); <span class="comment">// (1)</span></div><div class="line">  vertx.deployVerticle(worker, <span class="keyword">new</span> DeploymentOptions().setWorker(isWorker), r0 -&gt; &#123; <span class="comment">// (2)</span></div><div class="line">    <span class="keyword">if</span> (r0.succeeded()) &#123;</div><div class="line">      <span class="keyword">this</span>.on(<span class="string">"job_complete"</span>, msg -&gt; &#123;</div><div class="line">        <span class="keyword">long</span> dur = <span class="keyword">new</span> Job(((JsonObject) msg.body()).getJsonObject(<span class="string">"job"</span>)).getDuration();</div><div class="line">        client.incrby(RedisHelper.getKey(<span class="string">"stats:work-time"</span>), dur, r1 -&gt; &#123; <span class="comment">// (3)</span></div><div class="line">          <span class="keyword">if</span> (r1.failed())</div><div class="line">            r1.cause().printStackTrace();</div><div class="line">        &#125;);</div><div class="line">      &#125;);</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们创建一个<code>KueWorker</code>实例 (1)。我们将在稍后详细介绍<code>KueWorker</code>的实现。然后我们根据提供的配置来部署此<code>KueWorker</code> (2)。<code>processInternal</code>方法的第三个参数代表此<code>KueWorker</code>是否为worker verticle。如果部署成功，我们就监听<code>complete</code>事件。每当接收到<code>complete</code>事件的时候，我们获取收到的信息（处理任务消耗的时间），然后用<code>incrby</code>增加对应的工作时间 (3)。</p>
<p>再回到前面三个处理方法中。除了部署<code>KueWorker</code>以外，我们还调用了<code>setupTimers</code>方法，用于设定定时器以监测延时任务以及监测活动任务TTL。</p>
<h2 id="监测延时任务">监测延时任务</h2><p>Vert.x Kue支持延时任务，因此我们需要在任务延时时间到达时将任务“提升”至工作队列中等待处理。这个工作是在<code>checkJobPromotion</code>方法中实现的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">checkJobPromotion</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">int</span> timeout = config.getInteger(<span class="string">"job.promotion.interval"</span>, <span class="number">1000</span>); <span class="comment">// (1)</span></div><div class="line">  <span class="keyword">int</span> limit = config.getInteger(<span class="string">"job.promotion.limit"</span>, <span class="number">1000</span>); <span class="comment">// (2)</span></div><div class="line">  vertx.setPeriodic(timeout, l -&gt; &#123; <span class="comment">// (3)</span></div><div class="line">    client.zrangebyscore(RedisHelper.getKey(<span class="string">"jobs:DELAYED"</span>), String.valueOf(<span class="number">0</span>), String.valueOf(System.currentTimeMillis()),</div><div class="line">      <span class="keyword">new</span> RangeLimitOptions(<span class="keyword">new</span> JsonObject().put(<span class="string">"offset"</span>, <span class="number">0</span>).put(<span class="string">"count"</span>, limit)), r -&gt; &#123;  <span class="comment">// (4)</span></div><div class="line">        <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">          r.result().forEach(r1 -&gt; &#123;</div><div class="line">            <span class="keyword">long</span> id = Long.parseLong(RedisHelper.stripFIFO((String) r1));</div><div class="line">            <span class="keyword">this</span>.getJob(id).compose(jr -&gt; jr.get().inactive())  <span class="comment">// (5)</span></div><div class="line">              .setHandler(jr -&gt; &#123;</div><div class="line">                <span class="keyword">if</span> (jr.succeeded()) &#123;</div><div class="line">                  jr.result().emit(<span class="string">"promotion"</span>, jr.result().getId()); <span class="comment">// (6)</span></div><div class="line">                &#125; <span class="keyword">else</span> &#123;</div><div class="line">                  jr.cause().printStackTrace();</div><div class="line">                &#125;</div><div class="line">              &#125;);</div><div class="line">          &#125;);</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">          r.cause().printStackTrace();</div><div class="line">        &#125;</div><div class="line">      &#125;);</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们从配置中获取监测延时任务的间隔(<code>job.promotion.interval</code>，默认1000ms)以及提升数量阈值(<code>job.promotion.limit</code>，默认1000)。然后我们使用<code>vertx.setPeriodic</code>方法设一个周期性的定时器 (3)，每隔一段时间就从Redis中获取需要被提升的任务 (4)。这里我们通过<code>zrangebyscore</code>获取每个需要被提升任务的<code>id</code>。我们来看一下<code>zrangebyscore</code>方法的定义：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="function">RedisClient <span class="title">zrangebyscore</span><span class="params">(String key, String min, String max, RangeLimitOptions options, Handler&lt;AsyncResult&lt;JsonArray&gt;&gt; handler)</span></span>;</div></pre></td></tr></table></figure>
<ul>
<li><code>key</code>: 某个有序集合的key，即<code>vertx_kue:jobs:DELAYED</code></li>
<li><code>min</code> and <code>max</code>: 最小值以及最大值（按照某种模式）。这里<code>min</code>是<strong>0</strong>，而<code>max</code>是当前时间戳</li>
</ul>
<p>我们来回顾一下<code>Job</code>类中的<code>state</code>方法。当我们要把任务状态设为<code>DELAYED</code>的时候，我们将score设为<code>promote_at</code>时间：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">case</span> DELAYED:</div><div class="line">  client.transaction().zadd(RedisHelper.getKey(<span class="string">"jobs:"</span> + newState.name()),</div><div class="line">    <span class="keyword">this</span>.promote_at, <span class="keyword">this</span>.zid, _failure());</div></pre></td></tr></table></figure>
<p>因此我们将<code>max</code>设为当前时间(<code>System.currentTimeMillis()</code>)，只要当前时间超过需要提升的时间，这就说明此任务可以被提升了。</p>
<ul>
<li><code>options</code>: range和limit配置。这里我们需要指定<code>LIMIT</code>值所以我们用<code>new RangeLimitOptions(new JsonObject().put(&quot;offset&quot;, 0).put(&quot;count&quot;, limit)</code>创建了一个配置</li>
</ul>
<p><code>zrangebyscore</code>的结果是一个<code>JsonArray</code>，里面包含着所有等待提升任务的<code>zid</code>。获得结果后我们就将每个<code>zid</code>转换为<code>id</code>，然后分别获取对应的任务实体，最后对每个任务调用<code>inactive</code>方法来将任务状态设为<code>INACTIVE</code> (5)。如果任务成功提升至工作队列，我们就发送<code>promotion</code>事件 (6)。</p>
<h2 id="CallbackKue_-_提供多语言支持">CallbackKue - 提供多语言支持</h2><p>我们知道，Vert.x支持多种语言(如JS，Ruby)，因此如果能让我们的Vert.x Kue支持多种语言那当然是极好的！这没有问题～Vert.x Codegen可以处理含<code>@VertxGen</code>注解的异步接口，生成多语言版本。<code>@VertxGen</code>注解同样限制异步方法 —— 需要基于回调，因此我们设计了一个<code>CallbackKue</code>接口用于提供多语言支持。<code>CallbackKue</code>的设计非常简单，其实现复用了<code>Kue</code>和<code>jobService</code>的代码。大家可以直接看源码，一目了然，这里就不细说了。</p>
<p>注意要生成多语言版本的代码，需要添加相应的依赖。比如要生成Ruby版本的代码就要向<code>build.gradle</code>中添加<code>compile(&quot;io.vertx:vertx-lang-ruby:${vertxVersion}&quot;)</code>。</p>
<h1 id="KueWorker_-_任务在此处理">KueWorker - 任务在此处理</h1><p>好啦，我们已经对Vert.x Kue Core的几个核心部分有了大致的了解了，现在是时候探索一下任务处理的本源 - <code>KueWorker</code>了～</p>
<p>每一个worker都对应一个特定的任务类型，并且绑定着特定的处理函数(<code>Handler</code>)，所以我们需要在创建的时候指定它们。</p>
<h2 id="prepareAndStart方法">prepareAndStart方法</h2><p>在<code>KueWorker</code>中，我们使用<code>prepareAndStart</code>方法来准备要处理的任务并且开始处理任务的过程：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">prepareAndStart</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">this</span>.getJobFromBackend().setHandler(jr -&gt; &#123; <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">if</span> (jr.succeeded()) &#123;</div><div class="line">      <span class="keyword">if</span> (jr.result().isPresent()) &#123;</div><div class="line">        <span class="keyword">this</span>.job = jr.result().get(); <span class="comment">// (2)</span></div><div class="line">        process(); <span class="comment">// (3)</span></div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="keyword">this</span>.emitJobEvent(<span class="string">"error"</span>, <span class="keyword">null</span>, <span class="keyword">new</span> JsonObject().put(<span class="string">"message"</span>, <span class="string">"job_not_exist"</span>));</div><div class="line">        <span class="keyword">throw</span> <span class="keyword">new</span> IllegalStateException(<span class="string">"job not exist"</span>);</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="keyword">this</span>.emitJobEvent(<span class="string">"error"</span>, <span class="keyword">null</span>, <span class="keyword">new</span> JsonObject().put(<span class="string">"message"</span>, jr.cause().getMessage()));</div><div class="line">        jr.cause().printStackTrace();</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>代码比较直观。首先我们通过<code>getJobFromBackend</code>方法从Redis中按照优先级顺序获取任务 (1)。如果成功获取任务，我们就把获取到的任务保存起来 (2) 然后通过<code>process</code>方法处理任务 (3)。如果中间出现错误，我们需要发送<code>error</code>错误事件，其中携带错误信息。</p>
<h2 id="使用zpop按照优先级顺序获取任务">使用zpop按照优先级顺序获取任务</h2><p>我们来看一下我们是如何从Redis中按照优先级顺序获取任务实体的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> Future&lt;Optional&lt;Job&gt;&gt; getJobFromBackend() &#123;</div><div class="line">  Future&lt;Optional&lt;Job&gt;&gt; future = Future.future();</div><div class="line">  client.blpop(RedisHelper.getKey(<span class="keyword">this</span>.type + <span class="string">":jobs"</span>), <span class="number">0</span>, r1 -&gt; &#123; <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">if</span> (r1.failed()) &#123;</div><div class="line">      client.lpush(RedisHelper.getKey(<span class="keyword">this</span>.type + <span class="string">":jobs"</span>), <span class="string">"1"</span>, r2 -&gt; &#123;</div><div class="line">        <span class="keyword">if</span> (r2.failed())</div><div class="line">          future.fail(r2.cause());</div><div class="line">      &#125;);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">this</span>.zpop(RedisHelper.getKey(<span class="string">"jobs:"</span> + <span class="keyword">this</span>.type + <span class="string">":INACTIVE"</span>)) <span class="comment">// (2)</span></div><div class="line">        .compose(kue::getJob) <span class="comment">// (3)</span></div><div class="line">        .setHandler(r -&gt; &#123;</div><div class="line">          <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">            future.complete(r.result());</div><div class="line">          &#125; <span class="keyword">else</span></div><div class="line">            future.fail(r.cause());</div><div class="line">        &#125;);</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">  <span class="keyword">return</span> future;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>之前我们已经了解到，每当我们保存一个任务的时候，我们都会向<code>vertx_kue:{type}:jobs</code>列表中插入一个新元素表示新的任务可供处理。因此这里我们通过<code>blpop</code>命令来等待可用的任务 (1)。一旦有任务可供处理，我们就利用<code>zpop</code>方法取出高优先级的任务的<code>zid</code> (2)。<code>zpop</code>命令是一个原子操作，用于从有序集合中弹出最小score值的元素。注意Redis没有实现<code>zpop</code>命令，因此我们需要自己实现。</p>
<p><a href="http://redis.io/topics/transactions#using-a-hrefcommandswatchwatcha-to-implement-zpop" target="_blank" rel="external">Redis官方文档</a>介绍了一种实现<code>zpop</code>命令的简单方法 - 利用 <code>WATCH</code>。这里我们利用另外一种思路实现<code>zpop</code>命令：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> Future&lt;Long&gt; <span class="title">zpop</span><span class="params">(String key)</span> </span>&#123;</div><div class="line">  Future&lt;Long&gt; future = Future.future();</div><div class="line">  client.transaction()</div><div class="line">    .multi(_failure())</div><div class="line">    .zrange(key, <span class="number">0</span>, <span class="number">0</span>, _failure())</div><div class="line">    .zremrangebyrank(key, <span class="number">0</span>, <span class="number">0</span>, _failure())</div><div class="line">    .exec(r -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">        JsonArray res = r.result();</div><div class="line">        <span class="keyword">if</span> (res.getJsonArray(<span class="number">0</span>).size() == <span class="number">0</span>) <span class="comment">// empty set</span></div><div class="line">          future.fail(<span class="keyword">new</span> IllegalStateException(<span class="string">"Empty zpop set"</span>));</div><div class="line">        <span class="keyword">else</span> &#123;</div><div class="line">          <span class="keyword">try</span> &#123;</div><div class="line">            future.complete(Long.parseLong(RedisHelper.stripFIFO(</div><div class="line">              res.getJsonArray(<span class="number">0</span>).getString(<span class="number">0</span>))));</div><div class="line">          &#125; <span class="keyword">catch</span> (Exception ex) &#123;</div><div class="line">            future.fail(ex);</div><div class="line">          &#125;</div><div class="line">        &#125;</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        future.fail(r.cause());</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">  <span class="keyword">return</span> future;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在我们的<code>zpop</code>的实现中，我们首先开始了一个事务块，然后依次执行<code>zrange</code>和<code>zremrangebyrank</code>命令。有关这些命令的详情我们就不细说了，可以参考<a href="http://redis.io/commands" target="_blank" rel="external">Redis官方文档</a>。然后我们提交事务，如果提交成功，我们会获得一个<code>JsonArray</code>类型的结果。正常情况下我们都可以通过<code>res.getJsonArray(0).getString(0)</code>获取到对应的<code>zid</code>值。获取到<code>zid</code>值以后我们就可以将其转换为任务的<code>id</code>了，最后我们将<code>id</code>置于<code>Future</code>内（因为<code>zpop</code>也是一个异步方法）。</p>
<p>接着回到<code>getJobFromBackend</code>方法中。获取到对应的<code>id</code>之后，我们就可以通过<code>Kue</code>的<code>getJob</code>函数获取任务实体了 (3)。由于<code>getJobFromBackend</code>也是一个异步方法，因此我们同样将结果置于<code>Future</code>中。</p>
<h2 id="真正的“处理”逻辑">真正的“处理”逻辑</h2><p>前边讲了那么多，都是在为处理任务做准备。。。不要着急，现在终于到了真正的“处理”逻辑咯！我们看一下<code>process</code>方法的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">process</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">long</span> curTime = System.currentTimeMillis();</div><div class="line">  <span class="keyword">this</span>.job.setStarted_at(curTime)</div><div class="line">    .set(<span class="string">"started_at"</span>, String.valueOf(curTime)) <span class="comment">// (1) set start time</span></div><div class="line">    .compose(Job::active) <span class="comment">// (2) set the job state to ACTIVE</span></div><div class="line">    .setHandler(r -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">        Job j = r.result();</div><div class="line">        <span class="comment">// emit start event</span></div><div class="line">        <span class="keyword">this</span>.emitJobEvent(<span class="string">"start"</span>, j, <span class="keyword">null</span>);  <span class="comment">// (3) emit job `start` event</span></div><div class="line">        <span class="comment">// (4) process logic invocation</span></div><div class="line">        <span class="keyword">try</span> &#123;</div><div class="line">          jobHandler.handle(j);</div><div class="line">        &#125; <span class="keyword">catch</span> (Exception ex) &#123;</div><div class="line">          j.done(ex);</div><div class="line">        &#125;</div><div class="line">        <span class="comment">// (5) consume the job done event</span></div><div class="line"></div><div class="line">        eventBus.consumer(Kue.workerAddress(<span class="string">"done"</span>, j), msg -&gt; &#123;</div><div class="line">          createDoneCallback(j).handle(Future.succeededFuture(</div><div class="line">            ((JsonObject) msg.body()).getJsonObject(<span class="string">"result"</span>)));</div><div class="line">        &#125;);</div><div class="line">        eventBus.consumer(Kue.workerAddress(<span class="string">"done_fail"</span>, j), msg -&gt; &#123;</div><div class="line">          createDoneCallback(j).handle(Future.failedFuture(</div><div class="line">            (String) msg.body()));</div><div class="line">        &#125;);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">          <span class="keyword">this</span>.emitJobEvent(<span class="string">"error"</span>, <span class="keyword">this</span>.job, <span class="keyword">new</span> JsonObject().put(<span class="string">"message"</span>, r.cause().getMessage()));</div><div class="line">          r.cause().printStackTrace();</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>到了最核心的函数了！首先我们先给开始时间赋值 (1) 然后将任务状态置为<code>ACTIVE</code> (2)。如果这两个操作成功的话，我们就向Event Bus发送任务开始(<code>start</code>)事件 (3)。接下来我们调用真正的处理逻辑 - 之前绑定的<code>jobHandler</code> (4)。如果处理过程中抛出异常的话，Vert.x Kue就会调用<code>job.done(ex)</code>方法发送<code>done_fail</code>内部事件来通知worker任务处理失败。但是似乎没有看到在哪里接收并处理<code>done</code>和<code>done_fail</code>事件呢？就在这 (5)！一旦Vert.x Kue接收到这两个事件，它就会调用对应的<code>handler</code>去进行任务完成或失败的相应操作。这里的<code>handler</code>是由<code>createDoneCallback</code>方法生成的：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> Handler&lt;AsyncResult&lt;JsonObject&gt;&gt; createDoneCallback(Job job) &#123;</div><div class="line">  <span class="keyword">return</span> r0 -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (job == <span class="keyword">null</span>) &#123;</div><div class="line">      <span class="keyword">return</span>;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">if</span> (r0.failed()) &#123;</div><div class="line">      <span class="keyword">this</span>.fail(r0.cause()); <span class="comment">// (1)</span></div><div class="line">      <span class="keyword">return</span>;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">long</span> dur = System.currentTimeMillis() - job.getStarted_at();</div><div class="line">    job.setDuration(dur)</div><div class="line">      .set(<span class="string">"duration"</span>, String.valueOf(dur)); <span class="comment">// (2)</span></div><div class="line">    JsonObject result = r0.result();</div><div class="line">    <span class="keyword">if</span> (result != <span class="keyword">null</span>) &#123;</div><div class="line">      job.setResult(result)</div><div class="line">        .set(<span class="string">"result"</span>, result.encodePrettily()); <span class="comment">// (3)</span></div><div class="line">    &#125;</div><div class="line"></div><div class="line">    job.complete().setHandler(r -&gt; &#123; <span class="comment">// (4)</span></div><div class="line">      <span class="keyword">if</span> (r.succeeded()) &#123;</div><div class="line">        Job j = r.result();</div><div class="line">        <span class="keyword">if</span> (j.isRemoveOnComplete()) &#123; <span class="comment">// (5)</span></div><div class="line">          j.remove();</div><div class="line">        &#125;</div><div class="line">        <span class="keyword">this</span>.emitJobEvent(<span class="string">"complete"</span>, j, <span class="keyword">null</span>); <span class="comment">// (6)</span></div><div class="line"></div><div class="line">        <span class="keyword">this</span>.prepareAndStart(); <span class="comment">// (7) 准备处理下一个任务</span></div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">  &#125;;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>任务处理有两种情况：完成和失败，因此我们先来看任务成功处理的情况。我们首先给任务的用时(<code>duration</code>)赋值 (2)，并且如果任务产生了结果，也给结果(<code>result</code>)赋值 (3)。然后我们调用<code>job.complete</code>方法将状态设置为<code>COMPLETE</code> (4)。如果成功的话，我们就检查<code>removeOnComplete</code>标志位 (5) 并决定是否将任务从Redis中移除。然后我们向Event Bus发送任务完成事件(<code>complete</code>)以及队列事件<code>job_complete</code> (6)。现在这个任务的处理过程已经结束了，worker需要准备处理下一个任务了，因此最后我们调用<code>prepareAndStart</code>方法准备处理下一个<code>Job</code>。</p>
<h2 id="处理失败了怎么办？">处理失败了怎么办？</h2><p>人生不如意事十之八九，任务处理过程中很可能会遇见各种各样的问题而失败。当任务处理失败时，我们调用<code>KueWorker</code>中的<code>fail</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">fail</span><span class="params">(Throwable ex)</span> </span>&#123;</div><div class="line">  job.failedAttempt(ex).setHandler(r -&gt; &#123; <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">if</span> (r.failed()) &#123;</div><div class="line">      <span class="keyword">this</span>.error(r.cause(), job); <span class="comment">// (2)</span></div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      Job res = r.result();</div><div class="line">      <span class="keyword">if</span> (res.hasAttempts()) &#123; <span class="comment">// (3)</span></div><div class="line">        <span class="keyword">this</span>.emitJobEvent(<span class="string">"failed_attempt"</span>, job, <span class="keyword">new</span> JsonObject().put(<span class="string">"message"</span>, ex.getMessage()));</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="keyword">this</span>.emitJobEvent(<span class="string">"failed"</span>, job, <span class="keyword">new</span> JsonObject().put(<span class="string">"message"</span>, ex.getMessage())); <span class="comment">// (4)</span></div><div class="line">      &#125;</div><div class="line">      prepareAndStart(); <span class="comment">// (5)</span></div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>面对失败时，我们首先通过<code>failedAttempt</code>方法尝试从错误中恢复 (1)。如果恢复失败（比如没有重试次数了）就向Event Bus发送<code>error</code>队列事件 (2)。如果恢复成功，我们就根据是否还有剩余重试次数来发送对应的事件(<code>failed</code>或者<code>failed_attempt</code>)。搞定错误以后，worker同样需要准备处理下一个任务了，因此最后我们调用<code>prepareAndStart</code>方法准备处理下一个<code>Job</code> (5)。</p>
<p>这就是<code>KueWorker</code>的全部实现，是不是很有趣呢？看了这么久的代码也有些累了，下面是时候来写个Kue应用跑一下咯～</p>
<h1 id="展示时间！">展示时间！</h1><p>在<code>io.vertx.blueprint.kue.example</code>包下(<code>kue-example</code>子工程)创建一个<code>LearningVertxVerticle</code>类，然后编写如下代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.kue.example;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.kue.Kue;</div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.kue.queue.Job;</div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.kue.queue.Priority;</div><div class="line"><span class="keyword">import</span> io.vertx.core.AbstractVerticle;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.JsonObject;</div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">LearningVertxVerticle</span> <span class="keyword">extends</span> <span class="title">AbstractVerticle</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">start</span><span class="params">()</span> <span class="keyword">throws</span> Exception </span>&#123;</div><div class="line">    <span class="comment">// 创建工作队列</span></div><div class="line">    Kue kue = Kue.createQueue(vertx, config());</div><div class="line"></div><div class="line">    <span class="comment">// 监听全局错误事件</span></div><div class="line">    kue.on(<span class="string">"error"</span>, message -&gt;</div><div class="line">      System.out.println(<span class="string">"[Global Error] "</span> + message.body()));</div><div class="line"></div><div class="line">    JsonObject data = <span class="keyword">new</span> JsonObject()</div><div class="line">      .put(<span class="string">"title"</span>, <span class="string">"Learning Vert.x"</span>)</div><div class="line">      .put(<span class="string">"content"</span>, <span class="string">"core"</span>);</div><div class="line"></div><div class="line">    <span class="comment">// 准备学习Vert.x，爽！</span></div><div class="line">    Job j = kue.createJob(<span class="string">"learn vertx"</span>, data)</div><div class="line">      .priority(Priority.HIGH)</div><div class="line">      .onComplete(r -&gt; &#123; <span class="comment">// 完成任务事件监听</span></div><div class="line">        System.out.println(<span class="string">"Feeling: "</span> + r.getResult().getString(<span class="string">"feeling"</span>, <span class="string">"none"</span>));</div><div class="line">    &#125;).onFailure(r -&gt; &#123; <span class="comment">// 任务失败事件监听</span></div><div class="line">        System.out.println(<span class="string">"eee...so difficult..."</span>);</div><div class="line">    &#125;).onProgress(r -&gt; &#123; <span class="comment">// 任务进度变更事件监听</span></div><div class="line">        System.out.println(<span class="string">"I love this! My progress =&gt; "</span> + r);</div><div class="line">      &#125;);</div><div class="line"></div><div class="line">    <span class="comment">// 保存任务</span></div><div class="line">    j.save().setHandler(r0 -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (r0.succeeded()) &#123;</div><div class="line">        <span class="comment">// 开始学习！</span></div><div class="line">        kue.processBlocking(<span class="string">"learn vertx"</span>, <span class="number">1</span>, job -&gt; &#123;</div><div class="line">          job.progress(<span class="number">10</span>, <span class="number">100</span>);</div><div class="line">          <span class="comment">// 3秒速成</span></div><div class="line">          vertx.setTimer(<span class="number">3000</span>, r1 -&gt; &#123;</div><div class="line">            job.setResult(<span class="keyword">new</span> JsonObject().put(<span class="string">"feeling"</span>, <span class="string">"amazing and wonderful!"</span>)) <span class="comment">// 结果</span></div><div class="line">              .done(); <span class="comment">// 完成啦！</span></div><div class="line">          &#125;);</div><div class="line">        &#125;);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        System.err.println(<span class="string">"Wow, something happened: "</span> + r0.cause().getMessage());</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>通常情况下，一个Vert.x Kue应用可以分为几部分：创建工作队列、创建任务、保存任务以及处理任务。我们推荐开发者把应用写成<code>Verticle</code>的形式。</p>
<p>在这个例子中，我们要模拟一个学习Vert.x的任务！首先我们通过<code>Kue.createQueue</code>方法创建一个工作队列并且通过<code>on(error, handler)</code>方法监听全局错误(<code>error</code>)事件。接着我们通过<code>kue.createJob</code>方法创建学习任务，将优先级设定为<code>HIGH</code>，并且监听<code>complete</code>、<code>failed</code>以及<code>progress</code>事件。然后我们需要保存任务，保存完毕以后我们就可以通过<code>processBlocking</code>方法来执行耗时任务了。在处理逻辑中，我们首先通过<code>job.progress</code>方法将进度设为<code>10</code>，然后使用<code>vertx.setTimer</code>方法设一个3秒的定时器，定时器时间到以后赋予结果并完成任务。</p>
<p>像往常一样，我们还需要在<code>build.gradle</code>中配置一下。我们需要将<code>kue-example</code>子工程中的<code>Main-Verticle</code>属性设为刚才写的<code>io.vertx.blueprint.kue.example.LearningVertxVerticle</code>：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">project</span>(<span class="string">"kue-example"</span>) &#123;</div><div class="line"></div><div class="line">  <span class="keyword">dependencies</span> &#123;</div><div class="line">    <span class="keyword">compile</span>(<span class="keyword">project</span>(<span class="string">":kue-core"</span>))</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  jar &#123;</div><div class="line">    archiveName = <span class="string">'vertx-blueprint-kue-example.jar'</span></div><div class="line">    <span class="keyword">from</span> &#123; <span class="keyword">configurations</span>.<span class="keyword">compile</span>.<span class="keyword">collect</span> &#123; it.isDirectory() ? it : zipTree(it) &#125; &#125;</div><div class="line">    manifest &#123;</div><div class="line">      attributes <span class="string">'Main-Class'</span>: <span class="string">'io.vertx.core.Launcher'</span></div><div class="line">      attributes <span class="string">'Main-Verticle'</span>: <span class="string">'io.vertx.blueprint.kue.example.LearningVertxVerticle'</span></div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>好了，到了展示时间了！打开终端，构建项目：</p>
<pre><code><span class="title">gradle</span> build
</code></pre><p>当然不要忘记运行Redis：</p>
<pre><code>redis-<span class="keyword">server</span>
</code></pre><p>然后我们先运行Vert.x Kue Core部分：</p>
<pre><code>java -jar kue-core<span class="regexp">/build/</span>libs<span class="regexp">/vertx-blueprint-kue-core.jar -cluster -ha -conf config/</span>config.json
</code></pre><p>然后再运行我们的实例：</p>
<pre><code>java -jar kue-example<span class="regexp">/build/</span>libs<span class="regexp">/vertx-blueprint-kue-example.jar -cluster -ha -conf config/</span>config.json
</code></pre><p>这时终端应该会依次显示输出：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">INFO: Kue Verticle is running...</div><div class="line">I love this! My progress =&gt; 10</div><div class="line">Feeling: amazing and wonderful!</div></pre></td></tr></table></figure>
<p>当然你也可以在Vert.x Kue的Web端查看任务情况。</p>
<h1 id="完成我们的探索之旅！">完成我们的探索之旅！</h1><p>棒极了！我们终于结束了我们的Vert.x Kue核心部分探索之旅～～！从这篇超长的教程中，你学到了如何利用Vert.x去开发一个基于消息的应用！太酷了！</p>
<p>如果想了解<code>kue-http</code>的实现，请移步<a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-http-tutorial/">Vert.x 蓝图 | Vert.x Kue 教程（Web部分）</a>。如果想了解更多的关于Vert.x Kue的特性，请移步<a href="https://github.com/sczyh30/vertx-blueprint-job-queue/blob/master/docs/zh-cn/vertx-kue-features.zh-cn.md" target="_blank" rel="external">Vert.x Kue 特性介绍</a>。</p>
<p>Vert.x能做的不仅仅是这些。想要了解更多的关于Vert.x的知识，请参考<a href="http://vertx.io/docs/" target="_blank" rel="external">Vert.x 官方文档</a> —— 这永远是资料最齐全的地方。</p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;本文章是 &lt;strong&gt;Vert.x 蓝图系列&lt;/strong&gt; 的第二篇教程。全系列：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-1-todo-back
    
    </summary>
    
      <category term="Asynchronous" scheme="http://www.sczyh30.com/categories/Asynchronous/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
      <category term="消息系统" scheme="http://www.sczyh30.com/tags/%E6%B6%88%E6%81%AF%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="分布式" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F/"/>
    
  </entry>
  
  <entry>
    <title>Vert.x Blueprint 系列教程(一) | 待办事项服务开发教程</title>
    <link href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-1-todo-backend-tutorial/"/>
    <id>http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-1-todo-backend-tutorial/</id>
    <published>2016-06-15T16:00:00.000Z</published>
    <updated>2016-11-24T06:07:28.000Z</updated>
    
    <content type="html"><![CDATA[<p>本文章是 <a href="http://vertx.io/blog/vert-x-blueprint-tutorials/" target="_blank" rel="external">Vert.x 蓝图系列</a> 的第一篇教程。全系列：</p>
<ul>
<li>Vert.x Blueprint 系列教程(一) | 待办事项服务开发教程</li>
<li><a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-2-vertx-kue-core-tutorial/">Vert.x Blueprint 系列教程(二) | 开发基于消息的应用 - Vert.x Kue 教程</a></li>
<li><a href="http://www.sczyh30.com/posts/Asynchronous/vertx-blueprint-3-micro-shop-microservice/">Vert.x Blueprint 系列教程(三) | Micro Shop 微服务应用实践</a></li>
</ul>
<h1 id="前言">前言</h1><p>在本教程中，我们会使用Vert.x来一步一步地开发一个REST风格的Web服务 - Todo Backend，你可以把它看作是一个简单的待办事项服务，我们可以自由添加或者取消各种待办事项。</p>
<p>通过本教程，你将会学习到以下的内容：</p>
<ul>
<li><strong>Vert.x</strong> 是什么，以及其基本设计思想</li>
<li><code>Verticle</code>是什么，以及如何使用<code>Verticle</code></li>
<li>如何用 <strong>Vert.x Web</strong> 来开发REST风格的Web服务</li>
<li><strong>异步编程风格</strong> 的应用</li>
<li>如何通过 Vert.x 的各种组件来进行数据的存储操作（如 <em>Redis</em> 和 <em>MySQL</em>）</li>
</ul>
<p>本教程是 <strong>Vert.x 蓝图系列</strong> 的第一篇教程，对应的Vert.x版本为<strong>3.3.3</strong>。本教程中的完整代码已托管至<a href="https://github.com/sczyh30/vertx-blueprint-todo-backend/tree/master" target="_blank" rel="external">GitHub</a>。</p>
<h1 id="踏入Vert-x之门">踏入Vert.x之门</h1><p>朋友，欢迎来到Vert.x的世界！初次听说Vert.x，你一定会非常好奇：这是啥？让我们来看一下Vert.x的官方解释：</p>
<blockquote>
<p>Vert.x is a tool-kit for building reactive applications on the JVM.</p>
</blockquote>
<p>(⊙o⊙)哦哦。。。翻译一下，Vert.x是一个在JVM上构建 <strong>响应式</strong> 应用的 <strong>工具集</strong> 。这个定义比较模糊，我们来简单解释一下：<strong>工具集</strong> 意味着Vert.x非常轻量，可以嵌入到你当前的应用中而不需要改变现有的结构；另一个重要的描述是 <strong>响应式</strong> —— Vert.x就是为构建响应式应用（系统）而设计的。响应式系统这个概念在 <a href="http://reactivemanifesto.org/" target="_blank" rel="external">Reactive Manifesto</a> 中有详细的定义。我们在这里总结4个要点：</p>
<ul>
<li>响应式的(Responsive)：一个响应式系统需要在 <em>合理</em> 的时间内处理请求。</li>
<li>弹性的(Resilient)：一个响应式系统必须在遇到 <em>异常</em> （崩溃，超时， <code>500</code> 错误等等）的时候保持响应的能力，所以它必须要为 <em>异常处理</em> 而设计。</li>
<li>可伸缩的(Elastic)：一个响应式系统必须在不同的负载情况下都要保持响应能力，所以它必须能伸能缩，并且可以利用最少的资源来处理负载。</li>
<li>消息驱动：一个响应式系统的各个组件之间通过 <strong>异步消息传递</strong> 来进行交互。</li>
</ul>
<p>Vert.x是 <strong>事件驱动的</strong>，同时也是非阻塞的。首先，我们来介绍 <strong>Event Loop</strong> 的概念。Event Loop是一组负责分发和处理事件的线程。注意，我们绝对不能去阻塞Event Loop线程，否则事件的处理过程会被阻塞，我们的应用就失去了响应能力。因此当我们在写Vert.x应用的时候，我们要时刻谨记 <strong>异步非阻塞开发模式</strong> 而不是传统的阻塞开发模式。我们将会在下面详细讲解异步非阻塞开发模式。</p>
<h1 id="我们的应用_-_待办事项服务">我们的应用 - 待办事项服务</h1><p>我们的应用是一个REST风格的待办事项服务，它非常简单，整个API其实就围绕着 <em>增删改查</em> 四种操作。所以我们可以设计以下的路由：</p>
<ul>
<li>添加待办事项: <code>POST /todos</code></li>
<li>获取某一待办事项: <code>GET /todos/:todoId</code></li>
<li>获取所有待办事项: <code>GET /todos</code></li>
<li>更新待办事项: <code>PATCH /todos/:todoId</code></li>
<li>删除某一待办事项: <code>DELETE /todos/:todoId</code></li>
<li>删除所有待办事项: <code>DELETE /todos</code></li>
</ul>
<p>注意我们这里不讨论REST风格API的设计规范（仁者见仁,智者见智），因此你也可以用你喜欢的方式去定义路由。</p>
<p>下面我们开始开发我们的项目！High起来～～～</p>
<h1 id="说干就干！">说干就干！</h1><p>Vert.x Core提供了一些较为底层的处理HTTP请求的功能，这对于Web开发来说不是很方便，因为我们通常不需要这么底层的功能，因此<a href="http://vertx.io/docs/vertx-web/java" target="_blank" rel="external">Vert.x Web</a>应运而生。Vert.x Web基于Vert.x Core，并且提供一组更易于创建Web应用的上层功能（如路由）。</p>
<h2 id="Gradle配置文件">Gradle配置文件</h2><p>首先我们先来创建我们的项目。在本教程中我们使用<strong>Gradle</strong>作为构建工具，当然你也可以使用其它诸如Maven之类的构建工具。我们的项目目录里需要有：</p>
<ol>
<li><code>src/main/java</code> 文件夹（源码目录）</li>
<li><code>src/test/java</code> 文件夹（测试目录）</li>
<li><code>build.gradle</code> 文件（Gradle配置文件）</li>
</ol>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">.</div><div class="line">├── build.gradle</div><div class="line">├── settings.gradle</div><div class="line">├── src</div><div class="line">│   ├── main</div><div class="line">│   │   └── java</div><div class="line">│   └── test</div><div class="line">│       └── java</div></pre></td></tr></table></figure>
<p>我们首先来创建 <code>build.gradle</code> 文件，这是Gradle对应的配置文件：</p>
<figure class="highlight groovy"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line">apply <span class="string">plugin:</span> <span class="string">'java'</span></div><div class="line"></div><div class="line">targetCompatibility = <span class="number">1.8</span></div><div class="line">sourceCompatibility = <span class="number">1.8</span></div><div class="line"></div><div class="line">repositories &#123;</div><div class="line">  jcenter()</div><div class="line">&#125;</div><div class="line"></div><div class="line">dependencies &#123;</div><div class="line"></div><div class="line">  compile <span class="string">"io.vertx:vertx-core:3.3.3"</span></div><div class="line">  compile <span class="string">'io.vertx:vertx-web:3.3.3'</span></div><div class="line"></div><div class="line">  testCompile <span class="string">'io.vertx:vertx-unit:3.3.3'</span></div><div class="line">  testCompile <span class="string">group:</span> <span class="string">'junit'</span>, <span class="string">name:</span> <span class="string">'junit'</span>, <span class="string">version:</span> <span class="string">'4.12'</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>你可能不是很熟悉Gradle，这不要紧。我们来解释一下：</p>
<ul>
<li>我们将 <code>targetCompatibility</code> 和 <code>sourceCompatibility</code> 这两个值都设为<strong>1.8</strong>，代表目标Java版本是Java 8。这非常重要，因为Vert.x就是基于Java 8构建的。</li>
<li>在<code>dependencies</code>中，我们声明了我们需要的依赖。<code>vertx-core</code> 和 <code>vert-web</code> 用于开发REST API。</li>
</ul>
<p>搞定<code>build.gradle</code>以后，我们开始写代码！</p>
<h2 id="待办事项对象">待办事项对象</h2><p>首先我们需要创建我们的数据实体对象 - <code>Todo</code> 实体。在<code>io.vertx.blueprint.todolist.entity</code>包下创建<code>Todo</code>类，并且编写以下代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div><div class="line">68</div><div class="line">69</div><div class="line">70</div><div class="line">71</div><div class="line">72</div><div class="line">73</div><div class="line">74</div><div class="line">75</div><div class="line">76</div><div class="line">77</div><div class="line">78</div><div class="line">79</div><div class="line">80</div><div class="line">81</div><div class="line">82</div><div class="line">83</div><div class="line">84</div><div class="line">85</div><div class="line">86</div><div class="line">87</div><div class="line">88</div><div class="line">89</div><div class="line">90</div><div class="line">91</div><div class="line">92</div><div class="line">93</div><div class="line">94</div><div class="line">95</div><div class="line">96</div><div class="line">97</div><div class="line">98</div><div class="line">99</div><div class="line">100</div><div class="line">101</div><div class="line">102</div><div class="line">103</div><div class="line">104</div><div class="line">105</div><div class="line">106</div><div class="line">107</div><div class="line">108</div><div class="line">109</div><div class="line">110</div><div class="line">111</div><div class="line">112</div><div class="line">113</div><div class="line">114</div><div class="line">115</div><div class="line">116</div><div class="line">117</div><div class="line">118</div><div class="line">119</div><div class="line">120</div><div class="line">121</div><div class="line">122</div><div class="line">123</div><div class="line">124</div><div class="line">125</div><div class="line">126</div><div class="line">127</div><div class="line">128</div><div class="line">129</div><div class="line">130</div><div class="line">131</div><div class="line">132</div><div class="line">133</div><div class="line">134</div><div class="line">135</div><div class="line">136</div><div class="line">137</div><div class="line">138</div><div class="line">139</div><div class="line">140</div><div class="line">141</div><div class="line">142</div><div class="line">143</div><div class="line">144</div><div class="line">145</div><div class="line">146</div><div class="line">147</div><div class="line">148</div><div class="line">149</div><div class="line">150</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.todolist.entity;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.codegen.annotations.DataObject;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.JsonObject;</div><div class="line"></div><div class="line"><span class="keyword">import</span> java.util.concurrent.atomic.AtomicInteger;</div><div class="line"></div><div class="line"></div><div class="line"><span class="meta">@DataObject</span>(generateConverter = <span class="keyword">true</span>)</div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">Todo</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> AtomicInteger acc = <span class="keyword">new</span> AtomicInteger(<span class="number">0</span>); <span class="comment">// counter</span></div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">int</span> id;</div><div class="line">  <span class="keyword">private</span> String title;</div><div class="line">  <span class="keyword">private</span> Boolean completed;</div><div class="line">  <span class="keyword">private</span> Integer order;</div><div class="line">  <span class="keyword">private</span> String url;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">Todo</span><span class="params">()</span> </span>&#123;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">Todo</span><span class="params">(Todo other)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.id = other.id;</div><div class="line">    <span class="keyword">this</span>.title = other.title;</div><div class="line">    <span class="keyword">this</span>.completed = other.completed;</div><div class="line">    <span class="keyword">this</span>.order = other.order;</div><div class="line">    <span class="keyword">this</span>.url = other.url;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">Todo</span><span class="params">(JsonObject obj)</span> </span>&#123;</div><div class="line">    TodoConverter.fromJson(obj, <span class="keyword">this</span>); <span class="comment">// 还未生成Converter的时候需要先注释掉，生成过后再取消注释</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">Todo</span><span class="params">(String jsonStr)</span> </span>&#123;</div><div class="line">    TodoConverter.fromJson(<span class="keyword">new</span> JsonObject(jsonStr), <span class="keyword">this</span>);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">Todo</span><span class="params">(<span class="keyword">int</span> id, String title, Boolean completed, Integer order, String url)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.id = id;</div><div class="line">    <span class="keyword">this</span>.title = title;</div><div class="line">    <span class="keyword">this</span>.completed = completed;</div><div class="line">    <span class="keyword">this</span>.order = order;</div><div class="line">    <span class="keyword">this</span>.url = url;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> JsonObject <span class="title">toJson</span><span class="params">()</span> </span>&#123;</div><div class="line">    JsonObject json = <span class="keyword">new</span> JsonObject();</div><div class="line">    TodoConverter.toJson(<span class="keyword">this</span>, json);</div><div class="line">    <span class="keyword">return</span> json;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">int</span> <span class="title">getId</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> id;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setId</span><span class="params">(<span class="keyword">int</span> id)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.id = id;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setIncId</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.id = acc.incrementAndGet();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">int</span> <span class="title">getIncId</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> acc.get();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">setIncIdWith</span><span class="params">(<span class="keyword">int</span> n)</span> </span>&#123;</div><div class="line">    acc.set(n);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> String <span class="title">getTitle</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> title;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setTitle</span><span class="params">(String title)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.title = title;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> Boolean <span class="title">isCompleted</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> getOrElse(completed, <span class="keyword">false</span>);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setCompleted</span><span class="params">(Boolean completed)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.completed = completed;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> Integer <span class="title">getOrder</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> getOrElse(order, <span class="number">0</span>);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setOrder</span><span class="params">(Integer order)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.order = order;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> String <span class="title">getUrl</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> url;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setUrl</span><span class="params">(String url)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.url = url;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">equals</span><span class="params">(Object o)</span> </span>&#123;</div><div class="line">    <span class="keyword">if</span> (<span class="keyword">this</span> == o) <span class="keyword">return</span> <span class="keyword">true</span>;</div><div class="line">    <span class="keyword">if</span> (o == <span class="keyword">null</span> || getClass() != o.getClass()) <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line"></div><div class="line">    Todo todo = (Todo) o;</div><div class="line"></div><div class="line">    <span class="keyword">if</span> (id != todo.id) <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">    <span class="keyword">if</span> (!title.equals(todo.title)) <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">    <span class="keyword">if</span> (completed != <span class="keyword">null</span> ? !completed.equals(todo.completed) : todo.completed != <span class="keyword">null</span>) <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">    <span class="keyword">return</span> order != <span class="keyword">null</span> ? order.equals(todo.order) : todo.order == <span class="keyword">null</span>;</div><div class="line"></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">int</span> <span class="title">hashCode</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">int</span> result = id;</div><div class="line">    result = <span class="number">31</span> * result + title.hashCode();</div><div class="line">    result = <span class="number">31</span> * result + (completed != <span class="keyword">null</span> ? completed.hashCode() : <span class="number">0</span>);</div><div class="line">    result = <span class="number">31</span> * result + (order != <span class="keyword">null</span> ? order.hashCode() : <span class="number">0</span>);</div><div class="line">    <span class="keyword">return</span> result;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> String <span class="title">toString</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> <span class="string">"Todo -&gt; &#123;"</span> +</div><div class="line">      <span class="string">"id="</span> + id +</div><div class="line">      <span class="string">", title='"</span> + title + <span class="string">'\''</span> +</div><div class="line">      <span class="string">", completed="</span> + completed +</div><div class="line">      <span class="string">", order="</span> + order +</div><div class="line">      <span class="string">", url='"</span> + url + <span class="string">'\''</span> +</div><div class="line">      <span class="string">'&#125;'</span>;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> &lt;T&gt; <span class="function">T <span class="title">getOrElse</span><span class="params">(T value, T defaultValue)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> value == <span class="keyword">null</span> ? defaultValue : value;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> Todo <span class="title">merge</span><span class="params">(Todo todo)</span> </span>&#123;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">new</span> Todo(id,</div><div class="line">      getOrElse(todo.title, title),</div><div class="line">      getOrElse(todo.completed, completed),</div><div class="line">      getOrElse(todo.order, order),</div><div class="line">      url);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们的 <code>Todo</code> 实体对象由序号<code>id</code>、标题<code>title</code>、次序<code>order</code>、地址<code>url</code>以及代表待办事项是否完成的一个标识<code>complete</code>组成。我们可以把它看作是一个简单的Java Bean。它可以被编码成JSON格式的数据，我们在后边会大量使用JSON（事实上，在Vert.x中JSON非常普遍）。同时注意到我们给<code>Todo</code>类加上了一个注解：<code>@DataObject</code>，这是用于生成JSON转换类的注解。</p>
<blockquote>
<p><code>@DataObject</code> 注解</p>
<p>被 <code>@DataObject</code> 注解的实体类需要满足以下条件：拥有一个拷贝构造函数以及一个接受一个 <code>JsonObject</code> 对象的构造函数。</p>
</blockquote>
<p>我们利用Vert.x Codegen来自动生成JSON转换类。我们需要在<code>build.gradle</code>中添加依赖：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">compileOnly <span class="string">'io.vertx:vertx-codegen:3.3.3'</span></div></pre></td></tr></table></figure>
<p>由于Vert.x Codegen仅在编译期生成代码，因此我们这里使用了<code>compileOnly</code>(相当于Maven中的<code>provided</code>。需要Gradle 2.12及以上版本)。同时，我们需要在<code>io.vertx.blueprint.todolist.entity</code>包中添加<code>package-info.java</code>文件来指引Vert.x Codegen生成代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="comment">/**</span></div><div class="line"> * Indicates that this module contains classes that need to be generated / processed.</div><div class="line"> */</div><div class="line"><span class="meta">@ModuleGen</span>(name = <span class="string">"vertx-blueprint-todo-entity"</span>, groupPackage = <span class="string">"io.vertx.blueprint.todolist.entity"</span>)</div><div class="line"><span class="keyword">package</span> io.vertx.blueprint.todolist.entity;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.codegen.annotations.ModuleGen;</div></pre></td></tr></table></figure>
<p>Vert.x Codegen本质上是一个注解处理器(annotation processing tool)，因此我们还需要在<code>build.gradle</code>中配置apt。往里面添加以下代码：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">task</span> annotationProcessing(type: JavaCompile, <span class="keyword">group</span>: <span class="string">'build'</span>) &#123;</div><div class="line">  <span class="keyword">source</span> = <span class="keyword">sourceSets</span>.main.java</div><div class="line">  <span class="keyword">classpath</span> = <span class="keyword">configurations</span>.<span class="keyword">compile</span> + <span class="keyword">configurations</span>.compileOnly</div><div class="line">  <span class="keyword">destinationDir</span> = <span class="keyword">project</span>.<span class="keyword">file</span>(<span class="string">'src/main/generated'</span>)</div><div class="line">  <span class="keyword">options</span>.compilerArgs = [</div><div class="line">    <span class="string">"-proc:only"</span>,</div><div class="line">    <span class="string">"-processor"</span>, <span class="string">"io.vertx.codegen.CodeGenProcessor"</span>,</div><div class="line">    <span class="string">"-AoutputDirectory=$&#123;destinationDir.absolutePath&#125;"</span></div><div class="line">  ]</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">sourceSets</span> &#123;</div><div class="line">  main &#123;</div><div class="line">    java &#123;</div><div class="line">      srcDirs += <span class="string">'src/main/generated'</span></div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line">compileJava &#123;</div><div class="line">  <span class="keyword">targetCompatibility</span> = <span class="number">1.8</span></div><div class="line">  <span class="keyword">sourceCompatibility</span> = <span class="number">1.8</span></div><div class="line"></div><div class="line">  dependsOn annotationProcessing</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这样，每次我们在编译项目的时候，Vert.x Codegen都会自动检测含有 <code>@DataObject</code> 注解的类并且根据配置生成JSON转换类。在本例中，我们应该会得到一个 <code>TodoConverter</code> 类，然后我们可以在<code>Todo</code>类中使用它。</p>
<h2 id="Verticle">Verticle</h2><p>下面我们来写我们的应用组件。在<code>io.vertx.blueprint.todolist.verticles</code>包中创建<code>SingleApplicationVerticle</code>类，并编写以下代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.todolist.verticles;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.core.AbstractVerticle;</div><div class="line"><span class="keyword">import</span> io.vertx.core.Future;</div><div class="line"><span class="keyword">import</span> io.vertx.redis.RedisClient;</div><div class="line"><span class="keyword">import</span> io.vertx.redis.RedisOptions;</div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">SingleApplicationVerticle</span> <span class="keyword">extends</span> <span class="title">AbstractVerticle</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String HTTP_HOST = <span class="string">"0.0.0.0"</span>;</div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String REDIS_HOST = <span class="string">"127.0.0.1"</span>;</div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">int</span> HTTP_PORT = <span class="number">8082</span>;</div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">int</span> REDIS_PORT = <span class="number">6379</span>;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> RedisClient redis;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">start</span><span class="params">(Future&lt;Void&gt; future)</span> <span class="keyword">throws</span> Exception </span>&#123;</div><div class="line">      <span class="comment">// TODO with start...</span></div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们的<code>SingleApplicationVerticle</code>类继承了<code>AbstractVerticle</code>抽象类。那么什么是 <code>Verticle</code> 呢？在Vert.x中，一个<code>Verticle</code>代表应用的某一组件。我们可以通过部署<code>Verticle</code>来运行这些组件。如果你了解 <strong>Actor</strong> 模型的话，你会发现它和Actor非常类似。</p>
<p>当<code>Verticle</code>被部署的时候，其<code>start</code>方法会被调用。我们注意到这里的<code>start</code>方法接受一个类型为<code>Future&lt;Void&gt;</code>的参数，这代表了这是一个异步的初始化方法。这里的<code>Future</code>代表着<code>Verticle</code>的初始化过程是否完成。你可以通过调用Future的<code>complete</code>方法来代表初始化过程完成，或者<code>fail</code>方法代表初始化过程失败。</p>
<p>现在我们<code>Verticle</code>的轮廓已经搞好了，那么下一步也就很明了了 - 创建HTTP Client并且配置路由，处理HTTP请求。</p>
<h1 id="Vert-x_Web与REST_API">Vert.x Web与REST API</h1><h2 id="创建HTTP服务端并配置路由">创建HTTP服务端并配置路由</h2><p>我们来给<code>start</code>方法加点东西：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">start</span><span class="params">(Future&lt;Void&gt; future)</span> <span class="keyword">throws</span> Exception </span>&#123;</div><div class="line">  initData();</div><div class="line">  Router router = Router.router(vertx); <span class="comment">// &lt;1&gt;</span></div><div class="line">  <span class="comment">// CORS support</span></div><div class="line">  Set&lt;String&gt; allowHeaders = <span class="keyword">new</span> HashSet&lt;&gt;();</div><div class="line">  allowHeaders.add(<span class="string">"x-requested-with"</span>);</div><div class="line">  allowHeaders.add(<span class="string">"Access-Control-Allow-Origin"</span>);</div><div class="line">  allowHeaders.add(<span class="string">"origin"</span>);</div><div class="line">  allowHeaders.add(<span class="string">"Content-Type"</span>);</div><div class="line">  allowHeaders.add(<span class="string">"accept"</span>);</div><div class="line">  Set&lt;HttpMethod&gt; allowMethods = <span class="keyword">new</span> HashSet&lt;&gt;();</div><div class="line">  allowMethods.add(HttpMethod.GET);</div><div class="line">  allowMethods.add(HttpMethod.POST);</div><div class="line">  allowMethods.add(HttpMethod.DELETE);</div><div class="line">  allowMethods.add(HttpMethod.PATCH);</div><div class="line"></div><div class="line">  router.route().handler(CorsHandler.create(<span class="string">"*"</span>) <span class="comment">// &lt;2&gt;</span></div><div class="line">    .allowedHeaders(allowHeaders)</div><div class="line">    .allowedMethods(allowMethods));</div><div class="line">  router.route().handler(BodyHandler.create()); <span class="comment">// &lt;3&gt;</span></div><div class="line"></div><div class="line">  <span class="comment">// <span class="doctag">TODO:</span>routes</span></div><div class="line"></div><div class="line">  vertx.createHttpServer() <span class="comment">// &lt;4&gt;</span></div><div class="line">    .requestHandler(router::accept)</div><div class="line">    .listen(PORT, HOST, result -&gt; &#123;</div><div class="line">        <span class="keyword">if</span> (result.succeeded())</div><div class="line">          future.complete();</div><div class="line">        <span class="keyword">else</span></div><div class="line">          future.fail(result.cause());</div><div class="line">      &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>(⊙o⊙)…一长串代码诶。。是不是看着很晕呢？我们来详细解释一下。</p>
<p>首先我们创建了一个 <code>Router</code> 实例 （1）。这里的<code>Router</code>代表路由器，相信做过Web开发的开发者们一定不会陌生。路由器负责将对应的HTTP请求分发至对应的处理逻辑（Handler）中。每个<code>Handler</code>负责处理请求并且写入回应结果。当HTTP请求到达时，对应的<code>Handler</code>会被调用。</p>
<p>然后我们创建了两个<code>Set</code>：<code>allowHeaders</code>和<code>allowMethods</code>，并且我们向里面添加了一些HTTP Header以及HTTP Method，然后我们给路由器绑定了一个<code>CorsHandler</code> （2）。<code>route()</code>方法（无参数）代表此路由匹配所有请求。这两个<code>Set</code>的作用是支持 <em>CORS</em>，因为我们的API需要开启CORS以便配合前端正常工作。有关CORS的详细内容我们就不在这里细说了，详情可以参考<a href="http://enable-cors.org/server.html" target="_blank" rel="external">这里</a>。我们这里只需要知道如何开启CORS支持即可。</p>
<p>接下来我们给路由器绑定了一个全局的<code>BodyHandler</code> （3），它的作用是处理HTTP请求正文并获取其中的数据。比如，在实现添加待办事项逻辑的时候，我们需要读取请求正文中的JSON数据，这时候我们就可以用<code>BodyHandler</code>。</p>
<p>最后，我们通过<code>vertx.createHttpServer()</code>方法来创建一个HTTP服务端 （4）。注意这个功能是Vert.x Core提供的底层功能之一。然后我们将我们的路由处理器绑定到服务端上，这也是Vert.x Web的核心。你可能不熟悉<code>router::accept</code>这样的表示，这是Java 8中的 <em>方法引用</em>，它相当于一个分发路由的<code>Handler</code>。当有请求到达时，Vert.x会调用<code>accept</code>方法。然后我们通过<code>listen</code>方法监听8082端口。因为创建服务端的过程可能失败，因此我们还需要给<code>listen</code>方法传递一个<code>Handler</code>来检查服务端是否创建成功。正如我们前面所提到的，我们可以使用<code>future.complete</code>来表示过程成功，或者用<code>future.fail</code>来表示过程失败。</p>
<p>到现在为止，我们已经创建好HTTP服务端了，但我们还没有见到任何的路由呢！不要着急，是时候去声明路由了！</p>
<h2 id="配置路由">配置路由</h2><p>下面我们来声明路由。正如我们之前提到的，我们的路由可以设计成这样：</p>
<ul>
<li>添加待办事项: <code>POST /todos</code></li>
<li>获取某一待办事项: <code>GET /todos/:todoId</code></li>
<li>获取所有待办事项: <code>GET /todos</code></li>
<li>更新待办事项: <code>PATCH /todos/:todoId</code></li>
<li>删除某一待办事项: <code>DELETE /todos/:todoId</code></li>
<li>删除所有待办事项: <code>DELETE /todos</code></li>
</ul>
<blockquote>
<p>路径参数</p>
<p>在URL中，我们可以通过<code>:name</code>的形式定义路径参数。当处理请求的时候，Vert.x会自动获取这些路径参数并允许我们访问它们。拿我们的路由举个例子，<code>/todos/19</code> 将 <code>todoId</code> 映射为 <code>19</code>。</p>
</blockquote>
<p>首先我们先在 <code>io.vertx.blueprint.todolist</code> 包下创建一个<code>Constants</code>类用于存储各种全局常量（当然也可以放到其对应的类中）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.todolist;</div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="class"><span class="keyword">class</span> <span class="title">Constants</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="title">Constants</span><span class="params">()</span> </span>&#123;&#125;</div><div class="line"></div><div class="line">  <span class="comment">/** API Route */</span></div><div class="line">  <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String API_GET = <span class="string">"/todos/:todoId"</span>;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String API_LIST_ALL = <span class="string">"/todos"</span>;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String API_CREATE = <span class="string">"/todos"</span>;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String API_UPDATE = <span class="string">"/todos/:todoId"</span>;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String API_DELETE = <span class="string">"/todos/:todoId"</span>;</div><div class="line">  <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String API_DELETE_ALL = <span class="string">"/todos"</span>;</div><div class="line"></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>然后我们将<code>start</code>方法中的<code>TODO</code>标识处替换为以下的内容：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="comment">// routes</span></div><div class="line">router.get(Constants.API_GET).handler(<span class="keyword">this</span>::handleGetTodo);</div><div class="line">router.get(Constants.API_LIST_ALL).handler(<span class="keyword">this</span>::handleGetAll);</div><div class="line">router.post(Constants.API_CREATE).handler(<span class="keyword">this</span>::handleCreateTodo);</div><div class="line">router.patch(Constants.API_UPDATE).handler(<span class="keyword">this</span>::handleUpdateTodo);</div><div class="line">router.delete(Constants.API_DELETE).handler(<span class="keyword">this</span>::handleDeleteOne);</div><div class="line">router.delete(Constants.API_DELETE_ALL).handler(<span class="keyword">this</span>::handleDeleteAll);</div></pre></td></tr></table></figure>
<p>代码很直观、明了。我们用对应的方法（如<code>get</code>,<code>post</code>,<code>patch</code>等等）将路由路径与路由器绑定，并且我们调用<code>handler</code>方法给每个路由绑定上对应的<code>Handler</code>，接受的<code>Handler</code>类型为<code>Handler&lt;RoutingContext&gt;</code>。这里我们分别绑定了六个方法引用，它们的形式都类似于这样：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleRequest</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    <span class="comment">// ...</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们将在稍后实现这六个方法，这也是我们待办事项服务逻辑的核心。</p>
<h2 id="异步方法模式">异步方法模式</h2><p>我们之前提到过，Vert.x是 <strong>异步、非阻塞的</strong> 。每一个异步的方法总会接受一个 <code>Handler</code> 参数作为回调函数，当对应的操作完成时会调用接受的<code>Handler</code>，这是异步方法的一种实现。还有一种等价的实现是返回<code>Future</code>对象：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">void</span> <span class="title">doAsync</span><span class="params">(A a, B b, Handler&lt;R&gt; handler)</span></span>;</div><div class="line"><span class="comment">// 这两种实现等价</span></div><div class="line"><span class="function">Future&lt;R&gt; <span class="title">doAsync</span><span class="params">(A a, B b)</span></span>;</div></pre></td></tr></table></figure>
<p>其中，<code>Future</code> 对象代表着一个操作的结果，这个操作可能还没有进行，可能正在进行，可能成功也可能失败。当操作完成时，<code>Future</code>对象会得到对应的结果。我们也可以通过<code>setHandler</code>方法给<code>Future</code>绑定一个<code>Handler</code>，当<code>Future</code>被赋予结果的时候，此<code>Handler</code>会被调用。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">Future&lt;R&gt; future = doAsync(A a, B b);</div><div class="line">future.setHandler(r -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (r.failed()) &#123;</div><div class="line">        <span class="comment">// 处理失败</span></div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="comment">// 操作结果</span></div><div class="line">    &#125;</div><div class="line">&#125;);</div></pre></td></tr></table></figure>
<p>Vert.x中大多数异步方法都是基于Handler的。而在本教程中，这两种异步模式我们都会接触到。</p>
<h2 id="待办事项逻辑实现">待办事项逻辑实现</h2><p>现在是时候来实现我们的待办事项业务逻辑了！这里我们使用 Redis 作为数据持久化存储。有关Redis的详细介绍请参照<a href="http://redis.io/" target="_blank" rel="external">Redis 官方网站</a>。Vert.x给我们提供了一个组件—— Vert.x-redis，允许我们以异步的形式操作Redis数据。</p>
<blockquote>
<p>如何安装Redis？</p>
<p>请参照Redis官方网站上详细的<a href="http://redis.io/download#installation" target="_blank" rel="external">安装指南</a>。</p>
</blockquote>
<h3 id="Vert-x_Redis">Vert.x Redis</h3><p>Vert.x Redis允许我们以异步的形式操作Redis数据。我们首先需要在<code>build.gradle</code>中添加以下依赖：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">compile</span> <span class="string">'io.vertx:vertx-redis-client:3.3.3'</span></div></pre></td></tr></table></figure>
<p>我们通过<code>RedisClient</code>对象来操作Redis中的数据，因此我们定义了一个类成员<code>redis</code>。在使用<code>RedisClient</code>之前，我们首先需要与Redis建立连接，并且需要配置（以<code>RedisOptions</code>的形式），后边我们再讲需要配置哪些东西。</p>
<p>我们来实现 <code>initData</code> 方法用于初始化 <code>RedisClient</code> 并且测试连接：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">initData</span><span class="params">()</span> </span>&#123;</div><div class="line">  RedisOptions config = <span class="keyword">new</span> RedisOptions()</div><div class="line">      .setHost(config().getString(<span class="string">"redis.host"</span>, REDIS_HOST)) <span class="comment">// redis host</span></div><div class="line">      .setPort(config().getInteger(<span class="string">"redis.port"</span>, REDIS_PORT)); <span class="comment">// redis port</span></div><div class="line"></div><div class="line">  <span class="keyword">this</span>.redis = RedisClient.create(vertx, config); <span class="comment">// create redis client</span></div><div class="line"></div><div class="line">  redis.hset(Constants.REDIS_TODO_KEY, <span class="string">"24"</span>, Json.encodePrettily( <span class="comment">// test connection</span></div><div class="line">    <span class="keyword">new</span> Todo(<span class="number">24</span>, <span class="string">"Something to do..."</span>, <span class="keyword">false</span>, <span class="number">1</span>, <span class="string">"todo/ex"</span>)), res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.failed()) &#123;</div><div class="line">      LOGGER.error(<span class="string">"Redis service is not running!"</span>);</div><div class="line">      res.cause().printStackTrace();</div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line"></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>当我们在加载Verticle的时候，我们会首先调用<code>initData</code>方法，这样可以保证<code>RedisClient</code>可以被正常创建。</p>
<h3 id="存储格式">存储格式</h3><p>我们知道，Redis支持各种格式的数据，并且支持多种方式存储（如<code>list</code>、<code>hash map</code>等）。这里我们将我们的待办事项存储在 <em>哈希表(map)</em> 中。我们使用待办事项的<code>id</code>作为key，JSON格式的待办事项数据作为value。同时，我们的哈希表本身也要有个key，我们把它命名为 <em>VERT_TODO</em>，并且存储到<code>Constants</code>类中：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String REDIS_TODO_KEY = <span class="string">"VERT_TODO"</span>;</div></pre></td></tr></table></figure>
<p>正如我们之前提到的，我们利用了生成的JSON数据转换类来实现<code>Todo</code>实体与JSON数据之间的转换（通过几个构造函数），在后面实现待办事项服务的时候可以广泛利用。</p>
<h3 id="获取/获取所有待办事项">获取/获取所有待办事项</h3><p>我们首先来实现获取待办事项的逻辑。正如我们之前所提到的，我们的处理逻辑方法需要接受一个<code>RoutingContext</code>类型的参数。我们看一下获取某一待办事项的逻辑方法(<code>handleGetTodo</code>)：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleGetTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  String todoID = context.request().getParam(<span class="string">"todoId"</span>); <span class="comment">// (1)</span></div><div class="line">  <span class="keyword">if</span> (todoID == <span class="keyword">null</span>)</div><div class="line">    sendError(<span class="number">400</span>, context.response()); <span class="comment">// (2)</span></div><div class="line">  <span class="keyword">else</span> &#123;</div><div class="line">    redis.hget(Constants.REDIS_TODO_KEY, todoID, x -&gt; &#123; <span class="comment">// (3)</span></div><div class="line">      <span class="keyword">if</span> (x.succeeded()) &#123;</div><div class="line">        String result = x.result();</div><div class="line">        <span class="keyword">if</span> (result == <span class="keyword">null</span>)</div><div class="line">          sendError(<span class="number">404</span>, context.response());</div><div class="line">        <span class="keyword">else</span> &#123;</div><div class="line">          context.response()</div><div class="line">            .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">            .end(result); <span class="comment">// (4)</span></div><div class="line">        &#125;</div><div class="line">      &#125; <span class="keyword">else</span></div><div class="line">        sendError(<span class="number">503</span>, context.response());</div><div class="line">    &#125;);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们先通过<code>getParam</code>方法获取路径参数<code>todoId</code> (1)。我们需要检测路径参数获取是否成功，如果不成功就返回 <code>400 Bad Request</code> 错误 (2)。这里我们写一个函数封装返回错误response的逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">sendError</span><span class="params">(<span class="keyword">int</span> statusCode, HttpServerResponse response)</span> </span>&#123;</div><div class="line">  response.setStatusCode(statusCode).end();</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里面，<code>end</code>方法是非常重要的。只有我们调用<code>end</code>方法时，对应的HTTP Response才能被发送回客户端。</p>
<p>再回到<code>handleGetTodo</code>方法中。如果我们成功获取到了<code>todoId</code>，我们可以通过<code>hget</code>操作从Redis中获取对应的待办事项 (3)。<code>hget</code>代表通过key从对应的哈希表中获取对应的value，我们来看一下<code>hget</code>函数的定义：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="function">RedisClient <span class="title">hget</span><span class="params">(String key, String field, Handler&lt;AsyncResult&lt;String&gt;&gt; handler)</span></span>;</div></pre></td></tr></table></figure>
<p>第一个参数<code>key</code>对应哈希表的key，第二个参数<code>field</code>代表待办事项的key，第三个参数代表当获取操作成功时对应的回调。在<code>Handler</code>中，我们首先检查操作是否成功，如果不成功就返回<code>503</code>错误。如果成功了，我们就可以获取操作的结果了。结果是<code>null</code>的话，说明Redis中没有对应的待办事项，因此我们返回<code>404 Not Found</code>代表不存在。如果结果存在，那么我们就可以通过<code>end</code>方法将其写入response中 (4)。注意到我们所有的RESTful API都返回JSON格式的数据，所以我们将<code>content-type</code>头设为<code>JSON</code>。</p>
<p>获取所有待办事项的逻辑<code>handleGetAll</code>与<code>handleGetTodo</code>大体上类似，但实现上有些许不同：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleGetAll</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  redis.hvals(Constants.REDIS_TODO_KEY, res -&gt; &#123; <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">if</span> (res.succeeded()) &#123;</div><div class="line">      String encoded = Json.encodePrettily(res.result().stream() <span class="comment">// (2)</span></div><div class="line">        .map(x -&gt; <span class="keyword">new</span> Todo((String) x))</div><div class="line">        .collect(Collectors.toList()));</div><div class="line">      context.response()</div><div class="line">        .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">        .end(encoded); <span class="comment">// (3)</span></div><div class="line">    &#125; <span class="keyword">else</span></div><div class="line">      sendError(<span class="number">503</span>, context.response());</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>这里我们通过<code>hvals</code>操作 (1) 来获取某个哈希表中的所有数据（以JSON数组的形式返回，即<code>JsonArray</code>对象）。在Handler中我们还是像之前那样先检查操作是否成功。如果成功的话我们就可以将结果写入response了。注意这里我们不能直接将返回的<code>JsonArray</code>写入response。想象一下返回的<code>JsonArray</code>包括着待办事项的key以及对应的JSON数据（字符串形式），因此此时每个待办事项对应的JSON数据都被转义了，所以我们需要先把这些转义过的JSON数据转换成实体对象，再重新编码。</p>
<p>我们这里采用了一种响应式编程思想的方法。首先我们了解到<code>JsonArray</code>类继承了<code>Iterable&lt;Object&gt;</code>接口（是不是感觉它很像<code>List</code>呢？），因此我们可以通过<code>stream</code>方法将其转化为<code>Stream</code>对象。注意这里的<code>Stream</code>可不是传统意义上讲的输入输出流(I/O stream)，而是数据流(data flow)。我们需要对数据流进行一系列的变换处理操作，这就是响应式编程的思想（也有点函数式编程的思想）。我们将数据流中的每个字符串数据转换为<code>Todo</code>实体对象，这个过程是通过<code>map</code>算子实现的。我们这里就不深入讨论<code>map</code>算子了，但它在函数式编程中非常重要。在<code>map</code>过后，我们通过<code>collect</code>方法将数据流“归约”成<code>List&lt;Todo&gt;</code>。现在我们就可以通过<code>Json.encodePrettily</code>方法对得到的list进行编码了，转换成JSON格式的数据。最后我们将转换后的结果写入到response中 (3)。</p>
<h3 id="创建待办事项">创建待办事项</h3><p>经过了上面两个业务逻辑实现的过程，你应该开始熟悉Vert.x了～现在我们来实现创建待办事项的逻辑：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleCreateTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  <span class="keyword">try</span> &#123;</div><div class="line">    <span class="keyword">final</span> Todo todo = wrapObject(<span class="keyword">new</span> Todo(context.getBodyAsString()), context);</div><div class="line">    <span class="keyword">final</span> String encoded = Json.encodePrettily(todo);</div><div class="line">    redis.hset(Constants.REDIS_TODO_KEY, String.valueOf(todo.getId()),</div><div class="line">      encoded, res -&gt; &#123;</div><div class="line">        <span class="keyword">if</span> (res.succeeded())</div><div class="line">          context.response()</div><div class="line">            .setStatusCode(<span class="number">201</span>)</div><div class="line">            .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">            .end(encoded);</div><div class="line">        <span class="keyword">else</span></div><div class="line">          sendError(<span class="number">503</span>, context.response());</div><div class="line">      &#125;);</div><div class="line">  &#125; <span class="keyword">catch</span> (DecodeException e) &#123;</div><div class="line">    sendError(<span class="number">400</span>, context.response());</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们通过<code>context.getBodyAsString()</code>方法来从请求正文中获取JSON数据并转换成<code>Todo</code>实体对象 (1)。这里我们包装了一个处理<code>Todo</code>实例的方法，用于给其添加必要的信息（如URL）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> Todo <span class="title">wrapObject</span><span class="params">(Todo todo, RoutingContext context)</span> </span>&#123;</div><div class="line">  <span class="keyword">int</span> id = todo.getId();</div><div class="line">  <span class="keyword">if</span> (id &gt; Todo.getIncId()) &#123;</div><div class="line">    Todo.setIncIdWith(id);</div><div class="line">  &#125; <span class="keyword">else</span> <span class="keyword">if</span> (id == <span class="number">0</span>)</div><div class="line">    todo.setIncId();</div><div class="line">  todo.setUrl(context.request().absoluteURI() + <span class="string">"/"</span> + todo.getId());</div><div class="line">  <span class="keyword">return</span> todo;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>对于没有ID（或者为默认ID）的待办事项，我们会给它分配一个ID。这里我们采用了自增ID的策略，通过<code>AtomicInteger</code>来实现。</p>
<p>然后我们通过<code>Json.encodePrettily</code>方法将我们的<code>Todo</code>实例再次编码成JSON格式的数据 (2)。接下来我们利用<code>hset</code>函数将待办事项实例插入到对应的哈希表中 (3)。如果插入成功，返回 <code>201</code> 状态码 (4)。</p>
<p>[NOTE 201 状态码? | 正如你所看到的那样，我们将状态码设为<code>201</code>，这代表<code>CREATED</code>（已创建）。另外，如果不指定状态码的话，Vert.x Web默认将状态码设为 <code>200 OK</code>。]</p>
<p>同时，我们接收到的HTTP请求首部可能格式不正确，因此我们需要在方法中捕获<code>DecodeException</code>异常。这样一旦捕获到<code>DecodeException</code>异常，我们就返回<code>400 Bad Request</code>状态码。</p>
<h3 id="更新待办事项">更新待办事项</h3><p>如果你想改变你的计划，你就需要更新你的待办事项。我们来实现更新待办事项的逻辑，它有点小复杂（或者说是，繁琐？）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div></pre></td><td class="code"><pre><div class="line"><span class="comment">// PATCH /todos/:todoId</span></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleUpdateTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  <span class="keyword">try</span> &#123;</div><div class="line">    String todoID = context.request().getParam(<span class="string">"todoId"</span>); <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">final</span> Todo newTodo = <span class="keyword">new</span> Todo(context.getBodyAsString()); <span class="comment">// (2)</span></div><div class="line">    <span class="comment">// handle error</span></div><div class="line">    <span class="keyword">if</span> (todoID == <span class="keyword">null</span> || newTodo == <span class="keyword">null</span>) &#123;</div><div class="line">      sendError(<span class="number">400</span>, context.response());</div><div class="line">      <span class="keyword">return</span>;</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    redis.hget(Constants.REDIS_TODO_KEY, todoID, x -&gt; &#123; <span class="comment">// (3)</span></div><div class="line">      <span class="keyword">if</span> (x.succeeded()) &#123;</div><div class="line">        String result = x.result();</div><div class="line">        <span class="keyword">if</span> (result == <span class="keyword">null</span>)</div><div class="line">          sendError(<span class="number">404</span>, context.response()); <span class="comment">// (4)</span></div><div class="line">        <span class="keyword">else</span> &#123;</div><div class="line">          Todo oldTodo = <span class="keyword">new</span> Todo(result);</div><div class="line">          String response = Json.encodePrettily(oldTodo.merge(newTodo)); <span class="comment">// (5)</span></div><div class="line">          redis.hset(Constants.REDIS_TODO_KEY, todoID, response, res -&gt; &#123; <span class="comment">// (6)</span></div><div class="line">            <span class="keyword">if</span> (res.succeeded()) &#123;</div><div class="line">              context.response()</div><div class="line">                .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">                .end(response); <span class="comment">// (7)</span></div><div class="line">            &#125;</div><div class="line">          &#125;);</div><div class="line">        &#125;</div><div class="line">      &#125; <span class="keyword">else</span></div><div class="line">        sendError(<span class="number">503</span>, context.response());</div><div class="line">    &#125;);</div><div class="line">  &#125; <span class="keyword">catch</span> (DecodeException e) &#123;</div><div class="line">    sendError(<span class="number">400</span>, context.response());</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>唔。。。一大长串代码诶。。。我们来看一下。首先我们从 <code>RoutingContext</code> 中获取路径参数 <code>todoId</code> (1)，这是我们想要更改待办事项对应的id。然后我们从请求正文中获取新的待办事项数据 (2)。这一步也有可能抛出 <code>DecodeException</code> 异常因此我们也需要去捕获它。要更新待办事项，我们需要先通过<code>hget</code>函数获取之前的待办事项 (3)，检查其是否存在。获取旧的待办事项之后，我们调用之前在<code>Todo</code>类中实现的<code>merge</code>方法将旧待办事项与新待办事项整合到一起 (5)，然后编码成JSON格式的数据。然后我们通过<code>hset</code>函数更新对应的待办事项 (6)（<code>hset</code>表示如果不存在就插入，存在就更新）。操作成功的话，返回 <code>200 OK</code> 状态。</p>
<p>这就是更新待办事项的逻辑～要有耐心哟，我们马上就要见到胜利的曙光了～下面我们来实现删除待办事项的逻辑。</p>
<h3 id="删除/删除全部待办事项">删除/删除全部待办事项</h3><p>删除待办事项的逻辑非常简单。我们利用<code>hdel</code>函数来删除某一待办事项，用<code>del</code>函数删掉所有待办事项（实际上是直接把那个哈希表给删了）。如果删除操作成功，返回<code>204 No Content</code> 状态。</p>
<p>这里直接给出代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleDeleteOne</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  String todoID = context.request().getParam(<span class="string">"todoId"</span>);</div><div class="line">  redis.hdel(Constants.REDIS_TODO_KEY, todoID, res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.succeeded())</div><div class="line">      context.response().setStatusCode(<span class="number">204</span>).end();</div><div class="line">    <span class="keyword">else</span></div><div class="line">      sendError(<span class="number">503</span>, context.response());</div><div class="line">  &#125;);</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleDeleteAll</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  redis.del(Constants.REDIS_TODO_KEY, res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.succeeded())</div><div class="line">      context.response().setStatusCode(<span class="number">204</span>).end();</div><div class="line">    <span class="keyword">else</span></div><div class="line">      sendError(<span class="number">503</span>, context.response());</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>啊哈！我们实现待办事项服务的Verticle已经完成咯～一颗赛艇！但是我们该如何去运行我们的<code>Verticle</code>呢？答案是，我们需要 <em>部署并运行</em> 我们的Verticle。还好Vert.x提供了一个运行Verticle的辅助工具：Vert.x Launcher，让我们来看看如何利用它。</p>
<h2 id="将应用与Vert-x_Launcher一起打包">将应用与Vert.x Launcher一起打包</h2><p>要通过Vert.x Launcher来运行Verticle，我们需要在<code>build.gradle</code>中配置一下：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line">jar &#123;</div><div class="line">  <span class="comment">// by default fat jar</span></div><div class="line">  archiveName = <span class="string">'vertx-blueprint-todo-backend-fat.jar'</span></div><div class="line">  <span class="keyword">from</span> &#123; <span class="keyword">configurations</span>.<span class="keyword">compile</span>.<span class="keyword">collect</span> &#123; it.isDirectory() ? it : zipTree(it) &#125; &#125;</div><div class="line">  manifest &#123;</div><div class="line">      attributes <span class="string">'Main-Class'</span>: <span class="string">'io.vertx.core.Launcher'</span></div><div class="line">      attributes <span class="string">'Main-Verticle'</span>: <span class="string">'io.vertx.blueprint.todolist.verticles.SingleApplicationVerticle'</span></div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<ul>
<li>在<code>jar</code>区块中，我们配置Gradle使其生成 <strong>fat-jar</strong>，并指定启动类。<em>fat-jar</em> 是一个给Vert.x应用打包的简便方法，它直接将我们的应用连同所有的依赖都给打包到jar包中去了，这样我们可以直接通过jar包运行我们的应用而不必再指定依赖的 <code>CLASSPATH</code></li>
<li>我们将<code>Main-Class</code>属性设为<code>io.vertx.core.Launcher</code>，这样就可以通过Vert.x Launcher来启动对应的Verticle了。另外我们需要将<code>Main-Verticle</code>属性设为我们想要部署的Verticle的类名（全名）。</li>
</ul>
<p>配置好了以后，我们就可以打包了：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">gradle build</div></pre></td></tr></table></figure>
<h2 id="运行我们的服务">运行我们的服务</h2><p>万事俱备，只欠东风。是时候运行我们的待办事项服务了！首先我们先启动Redis服务：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">redis-server</div></pre></td></tr></table></figure>
<p>然后运行服务：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">java -jar build/libs/vertx-blueprint-todo-backend-fat.jar</div></pre></td></tr></table></figure>
<p>如果没问题的话，你将会在终端中看到 <code>Succeeded in deploying verticle</code> 的字样。下面我们可以自由测试我们的API了，其中最简便的方法是借助 <a href="https://github.com/TodoBackend/todo-backend-js-spec" target="_blank" rel="external">todo-backend-js-spec</a> 来测试。</p>
<p>键入 <code>http://127.0.0.1:8082/todos</code>：</p>
<p><img src="https://github.com/sczyh30/vertx-blueprint-todo-backend/raw/master/docs/img/todo-test-input.png" alt=""></p>
<p>测试结果：</p>
<p><img src="https://github.com/sczyh30/vertx-blueprint-todo-backend/raw/master/docs/img/todo-test-result.png" alt=""></p>
<p>当然，我们也可以用其它工具，比如 <code>curl</code> ：</p>
<figure class="highlight"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div></pre></td><td class="code"><pre><div class="line">sczyh30@sczyh30-workshop:~$ curl http://127.0.0.1:8082/todos</div><div class="line">[ &#123;</div><div class="line">  "id" : 20578623,</div><div class="line">  "title" : "blah",</div><div class="line">  "completed" : false,</div><div class="line">  "order" : 95,</div><div class="line">  "url" : "http://127.0.0.1:8082/todos/20578623"</div><div class="line">&#125;, &#123;</div><div class="line">  "id" : 1744802607,</div><div class="line">  "title" : "blah",</div><div class="line">  "completed" : false,</div><div class="line">  "order" : 523,</div><div class="line">  "url" : "http://127.0.0.1:8082/todos/1744802607"</div><div class="line">&#125;, &#123;</div><div class="line">  "id" : 981337975,</div><div class="line">  "title" : "blah",</div><div class="line">  "completed" : false,</div><div class="line">  "order" : 95,</div><div class="line">  "url" : "http://127.0.0.1:8082/todos/981337975"</div><div class="line">&#125; ]</div></pre></td></tr></table></figure>
<h1 id="将服务与控制器分离">将服务与控制器分离</h1><p>啊哈～我们的待办事项服务已经可以正常运行了，但是回头再来看看 <code>SingleApplicationVerticle</code> 类的代码，你会发现它非常混乱，待办事项业务逻辑与控制器混杂在一起，让这个类非常的庞大，并且这也不利于我们服务的扩展。根据面向对象解耦的思想，我们需要将控制器部分与业务逻辑部分分离。</p>
<h2 id="用Future实现异步服务">用Future实现异步服务</h2><p>下面我们来设计我们的业务逻辑层。就像我们之前提到的那样，我们的服务需要是异步的，因此这些服务的方法要么需要接受一个<code>Handler</code>参数作为回调，要么需要返回一个<code>Future</code>对象。但是想象一下很多个<code>Handler</code>混杂在一起嵌套的情况，你会陷入 <em>回调地狱</em>，这是非常糟糕的。因此，这里我们用<code>Future</code>实现我们的待办事项服务。</p>
<p>在 <code>io.vertx.blueprint.todolist.service</code> 包下创建 <code>TodoService</code> 接口并且编写以下代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.todolist.service;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.todolist.entity.Todo;</div><div class="line"><span class="keyword">import</span> io.vertx.core.Future;</div><div class="line"></div><div class="line"><span class="keyword">import</span> java.util.List;</div><div class="line"><span class="keyword">import</span> java.util.Optional;</div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">TodoService</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="function">Future&lt;Boolean&gt; <span class="title">initData</span><span class="params">()</span></span>; <span class="comment">// 初始化数据（或数据库）</span></div><div class="line"></div><div class="line">  <span class="function">Future&lt;Boolean&gt; <span class="title">insert</span><span class="params">(Todo todo)</span></span>;</div><div class="line"></div><div class="line">  Future&lt;List&lt;Todo&gt;&gt; getAll();</div><div class="line"></div><div class="line">  Future&lt;Optional&lt;Todo&gt;&gt; getCertain(String todoID);</div><div class="line"></div><div class="line">  <span class="function">Future&lt;Todo&gt; <span class="title">update</span><span class="params">(String todoId, Todo newTodo)</span></span>;</div><div class="line"></div><div class="line">  <span class="function">Future&lt;Boolean&gt; <span class="title">delete</span><span class="params">(String todoId)</span></span>;</div><div class="line"></div><div class="line">  <span class="function">Future&lt;Boolean&gt; <span class="title">deleteAll</span><span class="params">()</span></span>;</div><div class="line"></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>注意到<code>getCertain</code>方法返回一个<code>Future&lt;Optional&lt;Todo&gt;&gt;</code>对象。那么<code>Optional</code>是啥呢？它封装了一个可能为空的对象。因为数据库里面可能没有与我们给定的<code>todoId</code>相对应的待办事项，查询的结果可能为空，因此我们给它包装上 <code>Optional</code>。<code>Optional</code> 可以避免万恶的 <code>NullPointerException</code>，并且它在函数式编程中用途特别广泛（在Haskell中对应 <strong>Maybe Monad</strong>）。</p>
<p>既然我们已经设计好我们的异步服务接口了，让我们来重构原先的Verticle吧！</p>
<h2 id="开始重构！">开始重构！</h2><p>我们创建一个新的Verticle。在 <code>io.vertx.blueprint.todolist.verticles</code> 包中创建 <code>TodoVerticle</code> 类，并编写以下代码：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div><div class="line">68</div><div class="line">69</div><div class="line">70</div><div class="line">71</div><div class="line">72</div><div class="line">73</div><div class="line">74</div><div class="line">75</div><div class="line">76</div><div class="line">77</div><div class="line">78</div><div class="line">79</div><div class="line">80</div><div class="line">81</div><div class="line">82</div><div class="line">83</div><div class="line">84</div><div class="line">85</div><div class="line">86</div><div class="line">87</div><div class="line">88</div><div class="line">89</div><div class="line">90</div><div class="line">91</div><div class="line">92</div><div class="line">93</div><div class="line">94</div><div class="line">95</div><div class="line">96</div><div class="line">97</div><div class="line">98</div><div class="line">99</div><div class="line">100</div><div class="line">101</div><div class="line">102</div><div class="line">103</div><div class="line">104</div><div class="line">105</div><div class="line">106</div><div class="line">107</div><div class="line">108</div><div class="line">109</div><div class="line">110</div><div class="line">111</div><div class="line">112</div><div class="line">113</div><div class="line">114</div><div class="line">115</div><div class="line">116</div><div class="line">117</div><div class="line">118</div><div class="line">119</div><div class="line">120</div><div class="line">121</div><div class="line">122</div><div class="line">123</div><div class="line">124</div><div class="line">125</div><div class="line">126</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.todolist.verticles;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.todolist.Constants;</div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.todolist.entity.Todo;</div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.todolist.service.TodoService;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.core.AbstractVerticle;</div><div class="line"><span class="keyword">import</span> io.vertx.core.AsyncResult;</div><div class="line"><span class="keyword">import</span> io.vertx.core.Future;</div><div class="line"><span class="keyword">import</span> io.vertx.core.Handler;</div><div class="line"><span class="keyword">import</span> io.vertx.core.http.HttpMethod;</div><div class="line"><span class="keyword">import</span> io.vertx.core.http.HttpServerResponse;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.DecodeException;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.Json;</div><div class="line"><span class="keyword">import</span> io.vertx.ext.web.Router;</div><div class="line"><span class="keyword">import</span> io.vertx.ext.web.RoutingContext;</div><div class="line"><span class="keyword">import</span> io.vertx.ext.web.handler.BodyHandler;</div><div class="line"><span class="keyword">import</span> io.vertx.ext.web.handler.CorsHandler;</div><div class="line"></div><div class="line"><span class="keyword">import</span> java.util.HashSet;</div><div class="line"><span class="keyword">import</span> java.util.Random;</div><div class="line"><span class="keyword">import</span> java.util.Set;</div><div class="line"><span class="keyword">import</span> java.util.function.Consumer;</div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">TodoVerticle</span> <span class="keyword">extends</span> <span class="title">AbstractVerticle</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String HOST = <span class="string">"0.0.0.0"</span>;</div><div class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">int</span> PORT = <span class="number">8082</span>;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> TodoService service;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">initData</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="comment">// TODO</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="meta">@Override</span></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">start</span><span class="params">(Future&lt;Void&gt; future)</span> <span class="keyword">throws</span> Exception </span>&#123;</div><div class="line">    Router router = Router.router(vertx);</div><div class="line">    <span class="comment">// CORS support</span></div><div class="line">    Set&lt;String&gt; allowHeaders = <span class="keyword">new</span> HashSet&lt;&gt;();</div><div class="line">    allowHeaders.add(<span class="string">"x-requested-with"</span>);</div><div class="line">    allowHeaders.add(<span class="string">"Access-Control-Allow-Origin"</span>);</div><div class="line">    allowHeaders.add(<span class="string">"origin"</span>);</div><div class="line">    allowHeaders.add(<span class="string">"Content-Type"</span>);</div><div class="line">    allowHeaders.add(<span class="string">"accept"</span>);</div><div class="line">    Set&lt;HttpMethod&gt; allowMethods = <span class="keyword">new</span> HashSet&lt;&gt;();</div><div class="line">    allowMethods.add(HttpMethod.GET);</div><div class="line">    allowMethods.add(HttpMethod.POST);</div><div class="line">    allowMethods.add(HttpMethod.DELETE);</div><div class="line">    allowMethods.add(HttpMethod.PATCH);</div><div class="line"></div><div class="line">    router.route().handler(BodyHandler.create());</div><div class="line">    router.route().handler(CorsHandler.create(<span class="string">"*"</span>)</div><div class="line">      .allowedHeaders(allowHeaders)</div><div class="line">      .allowedMethods(allowMethods));</div><div class="line"></div><div class="line">    <span class="comment">// routes</span></div><div class="line">    router.get(Constants.API_GET).handler(<span class="keyword">this</span>::handleGetTodo);</div><div class="line">    router.get(Constants.API_LIST_ALL).handler(<span class="keyword">this</span>::handleGetAll);</div><div class="line">    router.post(Constants.API_CREATE).handler(<span class="keyword">this</span>::handleCreateTodo);</div><div class="line">    router.patch(Constants.API_UPDATE).handler(<span class="keyword">this</span>::handleUpdateTodo);</div><div class="line">    router.delete(Constants.API_DELETE).handler(<span class="keyword">this</span>::handleDeleteOne);</div><div class="line">    router.delete(Constants.API_DELETE_ALL).handler(<span class="keyword">this</span>::handleDeleteAll);</div><div class="line"></div><div class="line">    vertx.createHttpServer()</div><div class="line">      .requestHandler(router::accept)</div><div class="line">      .listen(PORT, HOST, result -&gt; &#123;</div><div class="line">          <span class="keyword">if</span> (result.succeeded())</div><div class="line">            future.complete();</div><div class="line">          <span class="keyword">else</span></div><div class="line">            future.fail(result.cause());</div><div class="line">        &#125;);</div><div class="line"></div><div class="line">    initData();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleCreateTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    <span class="comment">// TODO</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleGetTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    <span class="comment">// TODO</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleGetAll</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    <span class="comment">// TODO</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleUpdateTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    <span class="comment">// TODO</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleDeleteOne</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    <span class="comment">// TODO</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleDeleteAll</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">     <span class="comment">// TODO</span></div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">sendError</span><span class="params">(<span class="keyword">int</span> statusCode, HttpServerResponse response)</span> </span>&#123;</div><div class="line">    response.setStatusCode(statusCode).end();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">badRequest</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    context.response().setStatusCode(<span class="number">400</span>).end();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">notFound</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    context.response().setStatusCode(<span class="number">404</span>).end();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">serviceUnavailable</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    context.response().setStatusCode(<span class="number">503</span>).end();</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">private</span> Todo <span class="title">wrapObject</span><span class="params">(Todo todo, RoutingContext context)</span> </span>&#123;</div><div class="line">    <span class="keyword">int</span> id = todo.getId();</div><div class="line">    <span class="keyword">if</span> (id &gt; Todo.getIncId()) &#123;</div><div class="line">      Todo.setIncIdWith(id);</div><div class="line">    &#125; <span class="keyword">else</span> <span class="keyword">if</span> (id == <span class="number">0</span>)</div><div class="line">      todo.setIncId();</div><div class="line">    todo.setUrl(context.request().absoluteURI() + <span class="string">"/"</span> + todo.getId());</div><div class="line">    <span class="keyword">return</span> todo;</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>很熟悉吧？这个<code>Verticle</code>的结构与我们之前的Verticle相类似，这里就不多说了。下面我们来利用我们之前编写的服务接口实现每一个控制器方法。</p>
<p>首先先实现 <code>initData</code> 方法，此方法用于初始化存储结构：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">initData</span><span class="params">()</span> </span>&#123;</div><div class="line">  <span class="keyword">final</span> String serviceType = config().getString(<span class="string">"service.type"</span>, <span class="string">"redis"</span>);</div><div class="line">  LOGGER.info(<span class="string">"Service Type: "</span> + serviceType);</div><div class="line">  <span class="keyword">switch</span> (serviceType) &#123;</div><div class="line">    <span class="keyword">case</span> <span class="string">"jdbc"</span>:</div><div class="line">      service = <span class="keyword">new</span> JdbcTodoService(vertx, config());</div><div class="line">      <span class="keyword">break</span>;</div><div class="line">    <span class="keyword">case</span> <span class="string">"redis"</span>:</div><div class="line">    <span class="keyword">default</span>:</div><div class="line">      RedisOptions config = <span class="keyword">new</span> RedisOptions()</div><div class="line">        .setHost(config().getString(<span class="string">"redis.host"</span>, <span class="string">"127.0.0.1"</span>))</div><div class="line">        .setPort(config().getInteger(<span class="string">"redis.port"</span>, <span class="number">6379</span>));</div><div class="line">      service = <span class="keyword">new</span> RedisTodoService(vertx, config);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  service.initData().setHandler(res -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (res.failed()) &#123;</div><div class="line">        LOGGER.error(<span class="string">"Persistence service is not running!"</span>);</div><div class="line">        res.cause().printStackTrace();</div><div class="line">      &#125;</div><div class="line">    &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们从配置中获取服务的类型，这里我们有两种类型的服务：<code>redis</code>和<code>jdbc</code>，默认是<code>redis</code>。接着我们会根据服务的类型以及对应的配置来创建服务。在这里，我们的配置都是从JSON格式的配置文件中读取，并通过Vert.x Launcher的<code>-conf</code>项加载。后面我们再讲要配置哪些东西。</p>
<p>接着我们给<code>service.initData()</code>方法返回的<code>Future</code>对象绑定了一个<code>Handler</code>，这个<code>Handler</code>将会在<code>Future</code>得到结果的时候被调用。一旦初始化过程失败，错误信息将会显示到终端上。</p>
<p>其它的方法实现也类似，这里就不详细解释了，直接放上代码，非常简洁明了：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div><div class="line">68</div><div class="line">69</div><div class="line">70</div><div class="line">71</div><div class="line">72</div><div class="line">73</div><div class="line">74</div><div class="line">75</div><div class="line">76</div><div class="line">77</div><div class="line">78</div><div class="line">79</div><div class="line">80</div><div class="line">81</div><div class="line">82</div><div class="line">83</div><div class="line">84</div><div class="line">85</div><div class="line">86</div><div class="line">87</div><div class="line">88</div><div class="line">89</div><div class="line">90</div><div class="line">91</div><div class="line">92</div><div class="line">93</div><div class="line">94</div><div class="line">95</div><div class="line">96</div><div class="line">97</div><div class="line">98</div><div class="line">99</div><div class="line">100</div><div class="line">101</div><div class="line">102</div><div class="line">103</div><div class="line">104</div><div class="line">105</div><div class="line">106</div><div class="line">107</div><div class="line">108</div><div class="line">109</div><div class="line">110</div><div class="line">111</div><div class="line">112</div><div class="line">113</div><div class="line">114</div></pre></td><td class="code"><pre><div class="line"><span class="comment">/**</span></div><div class="line"> * Wrap the result handler with failure handler (503 Service Unavailable)</div><div class="line"> */</div><div class="line"><span class="keyword">private</span> &lt;T&gt; Handler&lt;AsyncResult&lt;T&gt;&gt; resultHandler(RoutingContext context, Consumer&lt;T&gt; consumer) &#123;</div><div class="line">  <span class="keyword">return</span> res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.succeeded()) &#123;</div><div class="line">      consumer.accept(res.result());</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      serviceUnavailable(context);</div><div class="line">    &#125;</div><div class="line">  &#125;;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleCreateTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  <span class="keyword">try</span> &#123;</div><div class="line">    <span class="keyword">final</span> Todo todo = wrapObject(<span class="keyword">new</span> Todo(context.getBodyAsString()), context);</div><div class="line">    <span class="keyword">final</span> String encoded = Json.encodePrettily(todo);</div><div class="line"></div><div class="line">    service.insert(todo).setHandler(resultHandler(context, res -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (res) &#123;</div><div class="line">        context.response()</div><div class="line">          .setStatusCode(<span class="number">201</span>)</div><div class="line">          .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">          .end(encoded);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        serviceUnavailable(context);</div><div class="line">      &#125;</div><div class="line">    &#125;));</div><div class="line">  &#125; <span class="keyword">catch</span> (DecodeException e) &#123;</div><div class="line">    sendError(<span class="number">400</span>, context.response());</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleGetTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  String todoID = context.request().getParam(<span class="string">"todoId"</span>);</div><div class="line">  <span class="keyword">if</span> (todoID == <span class="keyword">null</span>) &#123;</div><div class="line">    sendError(<span class="number">400</span>, context.response());</div><div class="line">    <span class="keyword">return</span>;</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  service.getCertain(todoID).setHandler(resultHandler(context, res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (!res.isPresent())</div><div class="line">      notFound(context);</div><div class="line">    <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">final</span> String encoded = Json.encodePrettily(res.get());</div><div class="line">      context.response()</div><div class="line">        .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">        .end(encoded);</div><div class="line">    &#125;</div><div class="line">  &#125;));</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleGetAll</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  service.getAll().setHandler(resultHandler(context, res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res == <span class="keyword">null</span>) &#123;</div><div class="line">      serviceUnavailable(context);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">final</span> String encoded = Json.encodePrettily(res);</div><div class="line">      context.response()</div><div class="line">        .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">        .end(encoded);</div><div class="line">    &#125;</div><div class="line">  &#125;));</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleUpdateTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  <span class="keyword">try</span> &#123;</div><div class="line">    String todoID = context.request().getParam(<span class="string">"todoId"</span>);</div><div class="line">    <span class="keyword">final</span> Todo newTodo = <span class="keyword">new</span> Todo(context.getBodyAsString());</div><div class="line">    <span class="comment">// handle error</span></div><div class="line">    <span class="keyword">if</span> (todoID == <span class="keyword">null</span>) &#123;</div><div class="line">      sendError(<span class="number">400</span>, context.response());</div><div class="line">      <span class="keyword">return</span>;</div><div class="line">    &#125;</div><div class="line">    service.update(todoID, newTodo)</div><div class="line">      .setHandler(resultHandler(context, res -&gt; &#123;</div><div class="line">        <span class="keyword">if</span> (res == <span class="keyword">null</span>)</div><div class="line">          notFound(context);</div><div class="line">        <span class="keyword">else</span> &#123;</div><div class="line">          <span class="keyword">final</span> String encoded = Json.encodePrettily(res);</div><div class="line">          context.response()</div><div class="line">            .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">            .end(encoded);</div><div class="line">        &#125;</div><div class="line">      &#125;));</div><div class="line">  &#125; <span class="keyword">catch</span> (DecodeException e) &#123;</div><div class="line">    badRequest(context);</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">private</span> Handler&lt;AsyncResult&lt;Boolean&gt;&gt; deleteResultHandler(RoutingContext context) &#123;</div><div class="line">  <span class="keyword">return</span> res -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (res.succeeded()) &#123;</div><div class="line">      <span class="keyword">if</span> (res.result()) &#123;</div><div class="line">        context.response().setStatusCode(<span class="number">204</span>).end();</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        serviceUnavailable(context);</div><div class="line">      &#125;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      serviceUnavailable(context);</div><div class="line">    &#125;</div><div class="line">  &#125;;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleDeleteOne</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  String todoID = context.request().getParam(<span class="string">"todoId"</span>);</div><div class="line">  service.delete(todoID)</div><div class="line">    .setHandler(deleteResultHandler(context));</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleDeleteAll</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">  service.deleteAll()</div><div class="line">    .setHandler(deleteResultHandler(context));</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>是不是和之前的Verticle很相似呢？这里我们还封装了两个<code>Handler</code>生成器：<code>resultHandler</code> 和 <code>deleteResultHandler</code>。这两个生成器封装了一些重复的代码，可以减少代码量。</p>
<p>嗯。。。我们的新Verticle写好了，那么是时候去实现具体的业务逻辑了。这里我们会实现两个版本的业务逻辑，分别对应两种存储：<strong>Redis</strong> 和 <strong>MySQL</strong>。</p>
<h2 id="Vert-x-Redis版本的待办事项服务">Vert.x-Redis版本的待办事项服务</h2><p>之前我们已经实现过一遍Redis版本的服务了，因此你应该对其非常熟悉了。这里我们仅仅解释一个 <code>update</code> 方法，其它的实现都非常类似，代码可以在<a href="https://github.com/sczyh30/vertx-blueprint-todo-backend/blob/master/src/main/java/io/vertx/blueprint/todolist/service/RedisTodoService.java" target="_blank" rel="external">GitHub</a>上浏览。</p>
<h3 id="Monadic_Future">Monadic Future</h3><p>回想一下我们之前写的更新待办事项的逻辑，我们会发现它其实是由两个独立的操作组成 - <code>get</code> 和 <code>insert</code>（对于Redis来说）。所以呢，我们可不可以复用 <code>getCertain</code> 和 <code>insert</code> 这两个方法？当然了！因为<code>Future</code>是可组合的，因此我们可以将这两个方法返回的<code>Future</code>组合到一起。是不是非常方便呢？我们来编写此方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Todo&gt; <span class="title">update</span><span class="params">(String todoId, Todo newTodo)</span> </span>&#123;</div><div class="line">  <span class="keyword">return</span> <span class="keyword">this</span>.getCertain(todoId).compose(old -&gt; &#123; <span class="comment">// (1)</span></div><div class="line">    <span class="keyword">if</span> (old.isPresent()) &#123;</div><div class="line">      Todo fnTodo = old.get().merge(newTodo);</div><div class="line">      <span class="keyword">return</span> <span class="keyword">this</span>.insert(fnTodo)</div><div class="line">        .map(r -&gt; r ? fnTodo : <span class="keyword">null</span>); <span class="comment">// (2)</span></div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      <span class="keyword">return</span> Future.succeededFuture(); <span class="comment">// (3)</span></div><div class="line">    &#125;</div><div class="line">  &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先我们调用了<code>getCertain</code>方法，此方法返回一个<code>Future&lt;Optional&lt;Todo&gt;&gt;</code>对象。同时我们使用<code>compose</code>函数将此方法返回的<code>Future</code>与另一个<code>Future</code>进行组合（1），其中<code>compose</code>函数接受一个<code>T =&gt; Future&lt;U&gt;</code>类型的lambda。然后我们接着检查旧的待办事项是否存在，如果存在的话，我们将新的待办事项与旧的待办事项相融合，然后更新待办事项。注意到<code>insert</code>方法返回<code>Future&lt;Boolean&gt;</code>类型的<code>Future</code>，因此我们还需要对此Future的结果做变换，这个变换的过程是通过<code>map</code>函数实现的（2）。<code>map</code>函数接受一个<code>T =&gt; U</code>类型的lambda。如果旧的待办事项不存在，我们返回一个包含null的<code>Future</code>（3）。最后我们返回组合后的<code>Future</code>对象。</p>
<blockquote>
<p><code>Future</code> 的本质</p>
<p>在函数式编程中，<code>Future</code> 实际上是一种 <code>Monad</code>。有关<code>Monad</code>的理论较为复杂，这里就不进行阐述了。你可以简单地把它看作是一个可以进行变换(<code>map</code>)和组合(<code>compose</code>)的包装对象。我们把这种特性叫做 <strong>monadic</strong>。</p>
</blockquote>
<p>下面来实现MySQL版本的待办事项服务。</p>
<h2 id="Vert-x-JDBC版本的待办事项服务">Vert.x-JDBC版本的待办事项服务</h2><h3 id="JDBC_++_异步">JDBC ++ 异步</h3><p>我们使用Vert.x-JDBC和MySQL来实现JDBC版本的待办事项服务。我们知道，数据库操作都是阻塞操作，很可能会占用不少时间。而Vert.x-JDBC提供了一种异步操作数据库的模式，很神奇吧？所以，在传统JDBC代码下我们要执行SQL语句需要这样：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">String SQL = <span class="string">"SELECT * FROM todo"</span>;</div><div class="line"><span class="comment">// ...</span></div><div class="line">ResultSet rs = pstmt.executeQuery(SQL);</div></pre></td></tr></table></figure>
<p>而在Vert.x JDBC中，我们可以利用回调获取数据：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">connection.query(SQL, result -&gt; &#123;</div><div class="line">    <span class="comment">// do something with result...</span></div><div class="line">&#125;);</div></pre></td></tr></table></figure>
<p>这种异步操作可以有效避免对数据的等待。当数据获取成功时会自动调用回调函数来执行处理数据的逻辑。</p>
<h3 id="添加依赖">添加依赖</h3><p>首先我们需要向<code>build.gradle</code>文件中添加依赖：</p>
<figure class="highlight groovy"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">compile <span class="string">'io.vertx:vertx-jdbc-client:3.3.3'</span></div><div class="line">compile <span class="string">'mysql:mysql-connector-java:6.0.2'</span></div></pre></td></tr></table></figure>
<p>其中第二个依赖是MySQL的驱动，如果你想使用其他的数据库，你需要自行替换掉这个依赖。</p>
<h3 id="初始化JDBCClient">初始化JDBCClient</h3><p>在Vert.x JDBC中，我们需要从一个<code>JDBCClient</code>对象中获取数据库连接，因此我们来看一下如何创建<code>JDBCClient</code>实例。在<code>io.vertx.blueprint.todolist.service</code>包下创建<code>JdbcTodoService</code>类：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">package</span> io.vertx.blueprint.todolist.service;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.blueprint.todolist.entity.Todo;</div><div class="line"></div><div class="line"><span class="keyword">import</span> io.vertx.core.Future;</div><div class="line"><span class="keyword">import</span> io.vertx.core.Vertx;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.JsonArray;</div><div class="line"><span class="keyword">import</span> io.vertx.core.json.JsonObject;</div><div class="line"><span class="keyword">import</span> io.vertx.ext.jdbc.JDBCClient;</div><div class="line"><span class="keyword">import</span> io.vertx.ext.sql.SQLConnection;</div><div class="line"></div><div class="line"><span class="keyword">import</span> java.util.List;</div><div class="line"><span class="keyword">import</span> java.util.Optional;</div><div class="line"><span class="keyword">import</span> java.util.stream.Collectors;</div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">JdbcTodoService</span> <span class="keyword">implements</span> <span class="title">TodoService</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="keyword">private</span> <span class="keyword">final</span> Vertx vertx;</div><div class="line">  <span class="keyword">private</span> <span class="keyword">final</span> JsonObject config;</div><div class="line">  <span class="keyword">private</span> <span class="keyword">final</span> JDBCClient client;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">JdbcTodoService</span><span class="params">(JsonObject config)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>(Vertx.vertx(), config);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="function"><span class="keyword">public</span> <span class="title">JdbcTodoService</span><span class="params">(Vertx vertx, JsonObject config)</span> </span>&#123;</div><div class="line">    <span class="keyword">this</span>.vertx = vertx;</div><div class="line">    <span class="keyword">this</span>.config = config;</div><div class="line">    <span class="keyword">this</span>.client = JDBCClient.createShared(vertx, config);</div><div class="line">  &#125;</div><div class="line"></div><div class="line">  <span class="comment">// ...</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们使用<code>JDBCClient.createShared(vertx, config)</code>方法来创建一个<code>JDBCClient</code>实例，其中我们传入一个<code>JsonObject</code>对象作为配置。一般来说，我们需要配置以下的内容：</p>
<ul>
<li><em>url</em> - JDBC URL，比如 <code>jdbc:mysql://localhost/vertx_blueprint</code></li>
<li><em>driver_class</em> - JDBC驱动名称，比如 <code>com.mysql.cj.jdbc.Driver</code></li>
<li><em>user</em> - 数据库用户</li>
<li><em>password</em> - 数据库密码</li>
</ul>
<p>我们将会通过Vert.x Launcher从配置文件中读取此<code>JsonObject</code>。</p>
<p>现在我们已经创建了<code>JDBCClient</code>实例了，下面我们需要在MySQL中建这样一个表：</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> <span class="string">`todo`</span> (</div><div class="line">  <span class="string">`id`</span> <span class="built_in">INT</span>(<span class="number">11</span>) <span class="keyword">NOT</span> <span class="literal">NULL</span> AUTO_INCREMENT,</div><div class="line">  <span class="string">`title`</span> <span class="built_in">VARCHAR</span>(<span class="number">255</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</div><div class="line">  <span class="string">`completed`</span> TINYINT(<span class="number">1</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</div><div class="line">  <span class="string">`order`</span> <span class="built_in">INT</span>(<span class="number">11</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</div><div class="line">  <span class="string">`url`</span> <span class="built_in">VARCHAR</span>(<span class="number">255</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</div><div class="line">  PRIMARY <span class="keyword">KEY</span> (<span class="string">`id`</span>)</div><div class="line">)</div></pre></td></tr></table></figure>
<p>我们把要用到的数据库语句都存到服务类中（这里我们就不讨论如何设计表以及写SQL了）：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String SQL_CREATE = <span class="string">"CREATE TABLE IF NOT EXISTS `todo` (\n"</span> +</div><div class="line">  <span class="string">"  `id` int(11) NOT NULL AUTO_INCREMENT,\n"</span> +</div><div class="line">  <span class="string">"  `title` varchar(255) DEFAULT NULL,\n"</span> +</div><div class="line">  <span class="string">"  `completed` tinyint(1) DEFAULT NULL,\n"</span> +</div><div class="line">  <span class="string">"  `order` int(11) DEFAULT NULL,\n"</span> +</div><div class="line">  <span class="string">"  `url` varchar(255) DEFAULT NULL,\n"</span> +</div><div class="line">  <span class="string">"  PRIMARY KEY (`id`) )"</span>;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String SQL_INSERT = <span class="string">"INSERT INTO `todo` "</span> +</div><div class="line">  <span class="string">"(`id`, `title`, `completed`, `order`, `url`) VALUES (?, ?, ?, ?, ?)"</span>;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String SQL_QUERY = <span class="string">"SELECT * FROM todo WHERE id = ?"</span>;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String SQL_QUERY_ALL = <span class="string">"SELECT * FROM todo"</span>;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String SQL_UPDATE = <span class="string">"UPDATE `todo`\n"</span> +</div><div class="line">  <span class="string">"SET `id` = ?,\n"</span> +</div><div class="line">  <span class="string">"`title` = ?,\n"</span> +</div><div class="line">  <span class="string">"`completed` = ?,\n"</span> +</div><div class="line">  <span class="string">"`order` = ?,\n"</span> +</div><div class="line">  <span class="string">"`url` = ?\n"</span> +</div><div class="line">  <span class="string">"WHERE `id` = ?;"</span>;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String SQL_DELETE = <span class="string">"DELETE FROM `todo` WHERE `id` = ?"</span>;</div><div class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> String SQL_DELETE_ALL = <span class="string">"DELETE FROM `todo`"</span>;</div></pre></td></tr></table></figure>
<p>OK！一切工作准备就绪，下面我们来实现我们的JDBC版本的服务～</p>
<h3 id="实现JDBC版本的服务">实现JDBC版本的服务</h3><p>所有的获取连接、获取执行数据的操作都要在<code>Handler</code>中完成。比如我们可以这样获取数据库连接：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">client.getConnection(conn -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (conn.succeeded()) &#123;</div><div class="line">        <span class="keyword">final</span> SQLConnection connection = conn.result();</div><div class="line">        <span class="comment">// do something...</span></div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="comment">// handle failure</span></div><div class="line">      &#125;</div><div class="line">    &#125;);</div></pre></td></tr></table></figure>
<p>由于每一个数据库操作都需要获取数据库连接，因此我们来包装一个返回<code>Handler&lt;AsyncResult&lt;SQLConnection&gt;&gt;</code>的方法，在此回调中可以直接使用数据库连接，可以减少一些代码量：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">private</span> Handler&lt;AsyncResult&lt;SQLConnection&gt;&gt; connHandler(Future future, Handler&lt;SQLConnection&gt; handler) &#123;</div><div class="line">  <span class="keyword">return</span> conn -&gt; &#123;</div><div class="line">    <span class="keyword">if</span> (conn.succeeded()) &#123;</div><div class="line">      <span class="keyword">final</span> SQLConnection connection = conn.result();</div><div class="line">      handler.handle(connection);</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">      future.fail(conn.cause());</div><div class="line">    &#125;</div><div class="line">  &#125;;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>获取数据库连接以后，我们就可以对数据库进行各种操作了：</p>
<ul>
<li><code>query</code> : 执行查询（raw SQL）</li>
<li><code>queryWithParams</code> : 执行预编译查询（prepared statement）</li>
<li><code>updateWithParams</code> : 执行预编译DDL语句（prepared statement）</li>
<li><code>execute</code>: 执行任意SQL语句</li>
</ul>
<p>所有的方法都是异步的所以每个方法最后都接受一个<code>Handler</code>参数，我们可以在此<code>Handler</code>中获取结果并执行相应逻辑。</p>
<p>现在我们来编写初始化数据库表的<code>initData</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Boolean&gt; <span class="title">initData</span><span class="params">()</span> </span>&#123;</div><div class="line">  Future&lt;Boolean&gt; result = Future.future();</div><div class="line">  client.getConnection(connHandler(result, connection -&gt;</div><div class="line">    connection.execute(SQL_CREATE, create -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (create.succeeded()) &#123;</div><div class="line">        result.complete(<span class="keyword">true</span>);</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        result.fail(create.cause());</div><div class="line">      &#125;</div><div class="line">      connection.close();</div><div class="line">    &#125;)));</div><div class="line">  <span class="keyword">return</span> result;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>此方法仅会在Verticle初始化时被调用，如果<code>todo</code>表不存在的话就创建一下。注意，<strong>最后一定要关闭数据库连接</strong>。</p>
<p>下面我们来实现插入逻辑方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> Future&lt;Boolean&gt; <span class="title">insert</span><span class="params">(Todo todo)</span> </span>&#123;</div><div class="line">  Future&lt;Boolean&gt; result = Future.future();</div><div class="line">  client.getConnection(connHandler(result, connection -&gt; &#123;</div><div class="line">    connection.updateWithParams(SQL_INSERT, <span class="keyword">new</span> JsonArray().add(todo.getId())</div><div class="line">      .add(todo.getTitle())</div><div class="line">      .add(todo.isCompleted())</div><div class="line">      .add(todo.getOrder())</div><div class="line">      .add(todo.getUrl()), r -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (r.failed()) &#123;</div><div class="line">        result.fail(r.cause());</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        result.complete(<span class="keyword">true</span>);</div><div class="line">      &#125;</div><div class="line">      connection.close();</div><div class="line">    &#125;);</div><div class="line">  &#125;));</div><div class="line">  <span class="keyword">return</span> result;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们使用<code>updateWithParams</code>方法执行插入逻辑，并且传递了一个<code>JsonArray</code>变量作为预编译参数。这一点很重要，使用预编译语句可以有效防止SQL注入。</p>
<p>我们再来实现<code>getCertain</code>方法：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="keyword">public</span> Future&lt;Optional&lt;Todo&gt;&gt; getCertain(String todoID) &#123;</div><div class="line">  Future&lt;Optional&lt;Todo&gt;&gt; result = Future.future();</div><div class="line">  client.getConnection(connHandler(result, connection -&gt; &#123;</div><div class="line">    connection.queryWithParams(SQL_QUERY, <span class="keyword">new</span> JsonArray().add(todoID), r -&gt; &#123;</div><div class="line">      <span class="keyword">if</span> (r.failed()) &#123;</div><div class="line">        result.fail(r.cause());</div><div class="line">      &#125; <span class="keyword">else</span> &#123;</div><div class="line">        List&lt;JsonObject&gt; list = r.result().getRows();</div><div class="line">        <span class="keyword">if</span> (list == <span class="keyword">null</span> || list.isEmpty()) &#123;</div><div class="line">          result.complete(Optional.empty());</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">          result.complete(Optional.of(<span class="keyword">new</span> Todo(list.get(<span class="number">0</span>))));</div><div class="line">        &#125;</div><div class="line">      &#125;</div><div class="line">      connection.close();</div><div class="line">    &#125;);</div><div class="line">  &#125;));</div><div class="line">  <span class="keyword">return</span> result;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在这个方法里，当我们的查询语句执行以后，我们获得到了<code>ResultSet</code>实例作为查询的结果集。我们可以通过<code>getColumnNames</code>方法获取字段名称，通过<code>getResults</code>方法获取结果。这里我们通过<code>getRows</code>方法来获取结果集，结果集的类型为<code>List&lt;JsonObject&gt;</code>。</p>
<p>其余的几个方法：<code>getAll</code>, <code>update</code>, <code>delete</code> 以及 <code>deleteAll</code>都遵循上面的模式，这里就不多说了。你可以在<a href="https://github.com/sczyh30/vertx-blueprint-todo-backend/blob/master/src/main/java/io/vertx/blueprint/todolist/service/JdbcTodoService.java" target="_blank" rel="external">GitHub</a>上浏览完整的源代码。</p>
<p>重构完毕，我们来写待办事项服务对应的配置，然后再来运行！</p>
<h2 id="再来运行！">再来运行！</h2><p>首先我们在项目的根目录下创建一个 <code>config</code> 文件夹作为配置文件夹。我们在其中创建一个<code>config_jdbc.json</code>文件作为 <code>jdbc</code> 类型服务的配置：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">&#123;</div><div class="line">  <span class="attr">"service.type"</span>: <span class="string">"jdbc"</span>,</div><div class="line">  <span class="attr">"url"</span>: <span class="string">"jdbc:mysql://localhost/vertx_blueprint?characterEncoding=UTF-8&amp;useSSL=false"</span>,</div><div class="line">  <span class="attr">"driver_class"</span>: <span class="string">"com.mysql.cj.jdbc.Driver"</span>,</div><div class="line">  <span class="attr">"user"</span>: <span class="string">"vbpdb1"</span>,</div><div class="line">  <span class="attr">"password"</span>: <span class="string">"666666*"</span>,</div><div class="line">  <span class="attr">"max_pool_size"</span>: <span class="number">30</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>你需要根据自己的情况替换掉上述配置文件中相应的内容(如 <em>JDBC URL</em>，<em>JDBC 驱动</em> 等)。</p>
<p>再建一个<code>config.json</code>文件作为<code>redis</code>类型服务的配置（其它的项就用默认配置好啦）：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">&#123;</div><div class="line">  <span class="attr">"service.type"</span>: <span class="string">"redis"</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们的构建文件也需要更新咯～这里直接给出最终的<code>build.gradle</code>文件：</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div></pre></td><td class="code"><pre><div class="line">plugins &#123;</div><div class="line">  id <span class="string">'java'</span></div><div class="line">&#125;</div><div class="line"></div><div class="line">version <span class="string">'1.0'</span></div><div class="line"></div><div class="line">ext &#123;</div><div class="line">  vertxVersion = <span class="string">"3.3.3"</span></div><div class="line">&#125;</div><div class="line"></div><div class="line">jar &#123;</div><div class="line">  <span class="comment">// by default fat jar</span></div><div class="line">  archiveName = <span class="string">'vertx-blueprint-todo-backend-fat.jar'</span></div><div class="line">  <span class="keyword">from</span> &#123; <span class="keyword">configurations</span>.<span class="keyword">compile</span>.<span class="keyword">collect</span> &#123; it.isDirectory() ? it : zipTree(it) &#125; &#125;</div><div class="line">  manifest &#123;</div><div class="line">    attributes <span class="string">'Main-Class'</span>: <span class="string">'io.vertx.core.Launcher'</span></div><div class="line">    attributes <span class="string">'Main-Verticle'</span>: <span class="string">'io.vertx.blueprint.todolist.verticles.TodoVerticle'</span></div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">repositories</span> &#123;</div><div class="line">  jcenter()</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="comment">// compileOnly requires Gradle 2.12+</span></div><div class="line"><span class="keyword">task</span> annotationProcessing(type: JavaCompile, <span class="keyword">group</span>: <span class="string">'build'</span>) &#123;</div><div class="line">  <span class="keyword">source</span> = <span class="keyword">sourceSets</span>.main.java</div><div class="line">  <span class="keyword">classpath</span> = <span class="keyword">configurations</span>.<span class="keyword">compile</span> + <span class="keyword">configurations</span>.compileOnly</div><div class="line">  <span class="keyword">destinationDir</span> = <span class="keyword">project</span>.<span class="keyword">file</span>(<span class="string">'src/main/generated'</span>)</div><div class="line">  <span class="keyword">options</span>.compilerArgs = [</div><div class="line">    <span class="string">"-proc:only"</span>,</div><div class="line">    <span class="string">"-processor"</span>, <span class="string">"io.vertx.codegen.CodeGenProcessor"</span>,</div><div class="line">    <span class="string">"-AoutputDirectory=$&#123;destinationDir.absolutePath&#125;"</span></div><div class="line">  ]</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">sourceSets</span> &#123;</div><div class="line">  main &#123;</div><div class="line">    java &#123;</div><div class="line">      srcDirs += <span class="string">'src/main/generated'</span></div><div class="line">    &#125;</div><div class="line">  &#125;</div><div class="line">&#125;</div><div class="line"></div><div class="line">compileJava &#123;</div><div class="line">  <span class="keyword">targetCompatibility</span> = <span class="number">1.8</span></div><div class="line">  <span class="keyword">sourceCompatibility</span> = <span class="number">1.8</span></div><div class="line"></div><div class="line">  dependsOn annotationProcessing</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="keyword">dependencies</span> &#123;</div><div class="line">  <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-core:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-web:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-jdbc-client:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  <span class="keyword">compile</span>(<span class="string">"io.vertx:vertx-redis-client:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  compileOnly(<span class="string">"io.vertx:vertx-codegen:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  <span class="keyword">compile</span> <span class="string">'mysql:mysql-connector-java:6.0.2'</span></div><div class="line"></div><div class="line">  testCompile(<span class="string">"io.vertx:vertx-unit:$&#123;vertxVersion&#125;"</span>)</div><div class="line">  testCompile <span class="keyword">group</span>: <span class="string">'junit'</span>, name: <span class="string">'junit'</span>, version: <span class="string">'4.12'</span></div><div class="line">&#125;</div><div class="line"></div><div class="line"></div><div class="line"><span class="keyword">task</span> wrapper(type: Wrapper) &#123;</div><div class="line">  gradleVersion = <span class="string">'3.0'</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>好啦好啦，迫不及待了吧？～打开终端，构建我们的应用：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">gradle build</div></pre></td></tr></table></figure>
<p>然后我们可以运行Redis版本的待办事项服务：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">java -jar build/libs/vertx-blueprint-todo-backend-fat.jar -conf config/config.json</div></pre></td></tr></table></figure>
<p>我们也可以运行JDBC版本的待办事项服务：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">java -jar build/libs/vertx-blueprint-todo-backend-fat.jar -conf config/config_jdbc.json</div></pre></td></tr></table></figure>
<p>同样地，我们也可以使用<a href="https://github.com/TodoBackend/todo-backend-js-spec" target="_blank" rel="external">todo-backend-js-spec</a>来测试我们的API。由于我们的API设计没有改变，因此测试结果应该不会有变化。</p>
<p>我们也提供了待办事项服务对应的Docker Compose镜像构建文件，可以直接通过Docker来运行我们的待办事项服务。你可以在仓库的根目录下看到<a href="https://github.com/sczyh30/vertx-blueprint-todo-backend/blob/master/docker-compose.yml" target="_blank" rel="external">相应的配置文件</a>，并通过 <code>docker-compose up -- build</code> 命令来构建并运行。</p>
<p><img src="https://github.com/sczyh30/vertx-blueprint-todo-backend/raw/master/docs/img/vbptds-docker-compose-running.png" alt="Docker Compose"></p>
<h1 id="哈哈，成功了！">哈哈，成功了！</h1><p>哈哈，恭喜你完成了整个待办事项服务，是不是很开心？～在整个教程中，你应该学到了很多关于 <code>Vert.x Web</code>、 <code>Vert.x Redis</code> 和 <code>Vert.x JDBC</code> 的开发知识。当然，最重要的是，你会对Vert.x的 <strong>异步开发模式</strong> 有了更深的理解和领悟。</p>
<p>另外，Vert.x 蓝图系列已经发布至Vert.x官网：<a href="http://vertx.io/blog/vert-x-blueprint-tutorials/" target="_blank" rel="external">Vert.x Blueprint Tutorials</a>。其中<a href="http://sczyh30.github.io/vertx-blueprint-job-queue/cn/kue-core/index.html" target="_blank" rel="external">第二个Blueprint</a>是关于消息应用的，<a href="http://sczyh30.github.io/vertx-blueprint-microservice/cn/index.html" target="_blank" rel="external">第三个Blueprint</a>是关于微服务的，有兴趣的朋友可以参考后面几篇蓝图教程。</p>
<p>更多关于Vert.x的文章，请参考<a href="http://vertx.io/blog/archives/" target="_blank" rel="external">Blog on Vert.x Website</a>。官网的资料是最全面的 :-)</p>
<h1 id="来自其它框架？">来自其它框架？</h1><p>之前你可能用过其它的框架，比如Spring Boot。这一小节，我将会用类比的方式来介绍Vert.x Web的使用。</p>
<h2 id="来自Spring_Boot/Spring_MVC">来自Spring Boot/Spring MVC</h2><p>在Spring Boot中，我们通常在控制器(Controller)中来配置路由以及处理请求，比如：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@RestController</span></div><div class="line"><span class="meta">@ComponentScan</span></div><div class="line"><span class="meta">@EnableAutoConfiguration</span></div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">TodoController</span> </span>&#123;</div><div class="line"></div><div class="line">  <span class="meta">@Autowired</span></div><div class="line">  <span class="keyword">private</span> TodoService service;</div><div class="line"></div><div class="line">  <span class="meta">@RequestMapping</span>(method = RequestMethod.GET, value = <span class="string">"/todos/&#123;id&#125;"</span>)</div><div class="line">  <span class="function"><span class="keyword">public</span> Todo <span class="title">getCertain</span><span class="params">(@PathVariable(<span class="string">"id"</span>)</span> <span class="keyword">int</span> id) </span>&#123;</div><div class="line">    <span class="keyword">return</span> service.fetch(id);</div><div class="line">  &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>在Spring Boot中，我们使用 <code>@RequestMapping</code> 注解来配置路由，而在Vert.x Web中，我们是通过 <code>Router</code> 对象来配置路由的。并且因为Vert.x Web是异步的，我们会给每个路由绑定一个处理器（<code>Handler</code>）来处理对应的请求。</p>
<p>另外，在Vert.x Web中，我们使用 <code>end</code> 方法来向客户端发送HTTP response。相对地，在Spring Boot中我们直接在每个方法中返回结果作为response。</p>
<h2 id="来自Play_Framework_2">来自Play Framework 2</h2><p>如果之前用过Play Framework 2的话，你一定会非常熟悉异步开发模式。在Play Framework 2中，我们在 <code>routes</code> 文件中定义路由，类似于这样：</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><span class="type">GET</span>     /todos/:todoId      controllers.<span class="type">TodoController</span>.handleGetCertain(todoId: <span class="type">Int</span>)</div></pre></td></tr></table></figure>
<p>而在Vert.x Web中，我们通过<code>Router</code>对象来配置路由：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">router.get(<span class="string">"/todos/:todoId"</span>).handler(<span class="keyword">this</span>::handleGetCertain);</div></pre></td></tr></table></figure>
<p><code>this::handleGetCertain</code>是处理对应请求的方法引用（在Scala里可以把它看作是一个函数）。</p>
<p>Play Framework 2中的异步开发模式是基于<code>Future</code>的。每一个路由处理函数都返回一个<code>Action</code>对象（实质上是一个类型为<code>Request[A] =&gt; Result</code>的函数），我们在<code>Action.apply</code>(或<code>Action.async</code>)闭包中编写我们的处理逻辑，类似于这样：</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">handleGetCertain</span></span>(todoId: <span class="type">Int</span>): <span class="type">Action</span>[<span class="type">AnyContent</span>] = <span class="type">Action</span>.async &#123;</div><div class="line">    service.getCertain(todoId) map &#123; <span class="comment">// 服务返回的类型是 `Future[Option[Todo]]`</span></div><div class="line">        <span class="keyword">case</span> <span class="type">Some</span>(res) =&gt;</div><div class="line">            <span class="type">Ok</span>(<span class="type">Json</span>.toJson(res))</div><div class="line">        <span class="keyword">case</span> <span class="type">None</span> =&gt;</div><div class="line">            <span class="type">NotFound</span>()</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>而在Vert.x Web中，异步开发模式基本上都是基于回调的（当然也可以用Vert.x RxJava）。我们可以这么写：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">handleCreateTodo</span><span class="params">(RoutingContext context)</span> </span>&#123;</div><div class="line">    String todoId = context.request().getParam(<span class="string">"todoId"</span>); <span class="comment">// 获取Path Variable</span></div><div class="line">    service.getCertain(todoId).setHandler(r -&gt; &#123; <span class="comment">// 服务返回的类型是 `Future&lt;Optional&lt;Todo&gt;&gt;`</span></div><div class="line">        <span class="keyword">if</span> (r.succeeded) &#123;</div><div class="line">            Optional&lt;Todo&gt; res = r.result;</div><div class="line">            <span class="keyword">if</span> (res.isPresent()) &#123;</div><div class="line">                context.response()</div><div class="line">                    .putHeader(<span class="string">"content-type"</span>, <span class="string">"application/json"</span>)</div><div class="line">                    .end(Json.encodePrettily(res));</div><div class="line">            &#125; <span class="keyword">else</span> &#123;</div><div class="line">                sendError(<span class="number">404</span>, context.response()); <span class="comment">// NotFound(404)</span></div><div class="line">            &#125;</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">            sendError(<span class="number">503</span>, context.response());</div><div class="line">        &#125;</div><div class="line">    &#125;);</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<h2 id="想要使用其它持久化存储框架？">想要使用其它持久化存储框架？</h2><p>你可能想在Vert.x中使用其它的持久化存储框架或库，比如MyBatis ORM或者Jedis，这当然可以啦！Vert.x允许开发者整合任何其它的框架和库，但是像MyBatis ORM这种框架都是阻塞型的，可能会阻塞Event Loop线程，因此我们需要利用<code>blockingHandler</code>方法去执行阻塞的操作：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">router.get(<span class="string">"/todos/:todoId"</span>).blockingHandler(routingContext -&gt; &#123;</div><div class="line">            String todoID = routingContext.request().getParam(<span class="string">"todoId"</span>);</div><div class="line">            Todo res = service.fetchBlocking(todoID); <span class="comment">// 阻塞型</span></div><div class="line"></div><div class="line">            <span class="comment">// 做一些微小的工作</span></div><div class="line"></div><div class="line">            routingContext.next();</div><div class="line">        &#125;);</div></pre></td></tr></table></figure>
<p>Vert.x会使用Worker线程去执行<code>blockingHandler</code>方法(或者Worker Verticles)中的操作，因此不会阻塞Event Loop线程。</p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;本文章是 &lt;a href=&quot;http://vertx.io/blog/vert-x-blueprint-tutorials/&quot; target=&quot;_blank&quot; rel=&quot;external&quot;&gt;Vert.x 蓝图系列&lt;/a&gt; 的第一篇教程。全系列：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;V
    
    </summary>
    
      <category term="Asynchronous" scheme="http://www.sczyh30.com/categories/Asynchronous/"/>
    
    
      <category term="异步编程" scheme="http://www.sczyh30.com/tags/%E5%BC%82%E6%AD%A5%E7%BC%96%E7%A8%8B/"/>
    
      <category term="Vert.x" scheme="http://www.sczyh30.com/tags/Vert-x/"/>
    
      <category term="Java" scheme="http://www.sczyh30.com/tags/Java/"/>
    
      <category term="Web开发" scheme="http://www.sczyh30.com/tags/Web%E5%BC%80%E5%8F%91/"/>
    
  </entry>
  
  <entry>
    <title>基于 Quorum 投票机制的 Replica Control 算法</title>
    <link href="http://www.sczyh30.com/posts/Architecture/quorum-based-voting-for-replica-control/"/>
    <id>http://www.sczyh30.com/posts/Architecture/quorum-based-voting-for-replica-control/</id>
    <published>2016-05-24T16:00:00.000Z</published>
    <updated>2018-09-23T14:53:47.000Z</updated>
    
    <content type="html"><![CDATA[<p>在分布式系统中，一份数据可能有多份副本（冗余数据）。为了保证数据读写的正确性，同一时刻一份数据的多份副本只能用于读或用于写，而不能同时被超过两个访问对象并发读写。Quorum机制就可以保证这一点，我们来看一下它的思想。</p>
<p>一个分布式系统中，我们给每个数据副本都赋予一票。假设一共有 <strong>V</strong> 个数据副本，那么总共就有 <strong>V</strong> 个票数。每个操作必须要获得读票数(完成读操作所需要读取的最小副本数，read quorum,  <strong>V(r)</strong> )或写票数(完成写操作所需要读取的最小副本数write quorum,  <strong>V(w)</strong> )才能够对数据进行读或写。票数需要遵循以下规则：</p>
<ol>
<li>$V_{r} + V_{w} &gt; V$</li>
<li>$V_{w} &gt; \frac{V}{2}$</li>
</ol>
<p>第一条规则有两个作用：第一个作用是保证了一个数据不会被同时读写。当请求一个写操作时，它需要的得到 <strong>V(w)</strong> 读票数，而剩下的票数为 <strong>V - V(w) &lt; V(r)</strong>，因此不再允许读操作。请求读操作时也是同理；第二个作用是保证了强一致性。根据 <strong>鸽巢原理</strong>，写数据操作与读新数据操作之间是有重叠的，这就确保至少有一个读操作是可以读到最新数据的。</p>
<p>第二条规则保证了数据的串行化修改，同一个数据不能同时被两个写操作并发修改。</p>
<p>Quorum投票机制非常有用。比如一份数据在5个结点上存有副本，进行一次写操作的时候，必须等待五个结点的写操作都完成，整个写操作才返回（因为可以从任意结点读取）。这样会导致写操作负载太高，而有了Quorum机制以后，我们可以让写操作在至少3个结点上完成就可以返回，另外的结点可以等待后台同步，而读操作V(r)也需要大于 <strong>V-V(w)</strong> 才能确保至少一个读操作可以读到最新数据。</p>
<hr>
<h1 id="References">References</h1><ul>
<li><em>Weighted Voting for Replicated Data</em>, David K. Gifford, Stanford University and Xerox Palo Alto Research Center</li>
</ul>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;在分布式系统中，一份数据可能有多份副本（冗余数据）。为了保证数据读写的正确性，同一时刻一份数据的多份副本只能用于读或用于写，而不能同时被超过两个访问对象并发读写。Quorum机制就可以保证这一点，我们来看一下它的思想。&lt;/p&gt;
&lt;p&gt;一个分布式系统中，我们给每个数据副本都赋
    
    </summary>
    
      <category term="Architecture" scheme="http://www.sczyh30.com/categories/Architecture/"/>
    
    
      <category term="分布式系统" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="Quorum" scheme="http://www.sczyh30.com/tags/Quorum/"/>
    
  </entry>
  
  <entry>
    <title>Netflix Hystrix | 工作流程浅析 &amp;&amp; HystrixCircuitBreaker 源码分析</title>
    <link href="http://www.sczyh30.com/posts/Microservice/netflix-hystrix-how-it-works-summary/"/>
    <id>http://www.sczyh30.com/posts/Microservice/netflix-hystrix-how-it-works-summary/</id>
    <published>2016-05-23T16:00:00.000Z</published>
    <updated>2018-09-23T14:52:51.000Z</updated>
    
    <content type="html"><![CDATA[<blockquote>
<p>2017-12-04：待重新总结。。。</p>
</blockquote>
<p>这篇文章里我们来总结一下 Netflix Hystrix 的工作流程（版本为 1.4.x）。这是官方提供的流程图（来自 <a href="https://github.com/Netflix/Hystrix/wiki/How-it-Works" target="_blank" rel="external">GitHub</a>）：</p>
<p><img src="/assets/blog-img-k1/hystrix-command-flow-chart.png" alt="Netflix Hystrix Flow Chart"></p>
<h1 id="工作流程">工作流程</h1><p>我们来根据流程图来分析一下工作流程。</p>
<p>首先我们需要创建一个 <code>HystrixCommand</code> 或 <code>HystrixObservableCommand</code> 实例来代表向其它组件发出的操作请求（指令），然后通过相关的方法执行操作指令。这里有4个方法，前两个对应<code>HystrixCommand</code>，后两个对应<code>HystrixObservableCommand</code>：</p>
<ul>
<li><code>execute()</code>：阻塞型方法，返回单个结果（或者抛出异常）</li>
<li><code>queue()</code>：异步方法，返回一个 <code>Future</code> 对象，可以从中取出单个结果（或者抛出异常）</li>
<li><code>observe()</code> 和 <code>toObservable()</code> 都返回对应的 <code>Observable</code> 对象，代表（多个）操作结果。注意 <code>observe</code> 方法在调用的时候就开始执行对应的指令（hot observable 加了层 buffer 代理），而 <code>toObservable</code> 方法相当于是 <code>observe</code> 方法的lazy版本，当我们去 <code>subscribe</code> 的时候，对应的指令才会被执行并产生结果</li>
</ul>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">K             value   = command.execute();</div><div class="line">Future&lt;K&gt;     fValue  = command.queue();</div><div class="line">Observable&lt;K&gt; ohValue = command.observe();         <span class="comment">//hot observable</span></div><div class="line">Observable&lt;K&gt; ocValue = command.toObservable();    <span class="comment">//cold observable</span></div></pre></td></tr></table></figure>
<p>从底层实现来讲，<code>HystrixCommand</code>也是利用<code>Observable</code>实现的（看Hystrix源码的话可以发现里面大量使用了RxJava），尽管它只返回单个结果。<code>HystrixCommand</code>的<code>queue</code>方法实际上是调用了<code>toObservable().toBlocking().toFuture()</code>，而<code>execute</code>方法实际上是调用了<code>queue().get()</code>。</p>
<p>执行操作指令时，Hystrix首先会检查缓存内是否有对应指令的结果，如果有的话，将缓存的结果直接以<code>Observable</code>对象的形式返回。如果没有对应的缓存，Hystrix会检查Circuit Breaker的状态。如果Circuit Breaker的状态为开启状态，Hystrix将不会执行对应指令，而是直接进入失败处理状态（图中8 Fallback）。如果Circuit Breaker的状态为关闭状态，Hystrix会继续进行线程池、任务队列、信号量的检查（图中5），确认是否有足够的资源执行操作指令。如果资源满，Hystrix同样将不会执行对应指令并且直接进入失败处理状态。</p>
<p>如果资源充足，Hystrix将会执行操作指令。操作指令的调用最终都会到这两个方法：</p>
<ul>
<li><code>HystrixCommand.run()</code></li>
<li><code>HystrixObservableCommand.construct()</code></li>
</ul>
<p>如果执行指令的时间超时，执行线程会抛出<code>TimeoutException</code>异常。Hystrix会抛弃结果并直接进入失败处理状态。如果执行指令成功，Hystrix会进行一系列的数据记录，然后返回执行的结果。</p>
<p>同时，Hystrix会根据记录的数据来计算失败比率，一旦失败比率达到某一阈值将自动开启Circuit Breaker。</p>
<p>最后我们再来看一下Hystrix是如何处理失败的。如果我们在Command中实现了<code>HystrixCommand.getFallback()</code>方法（或<code>HystrixObservableCommand.resumeWithFallback()</code>方法，Hystrix会返回对应方法的结果。如果没有实现这些方法的话，从底层看Hystrix将会返回一个空的<code>Observable</code>对象，并且可以通过<code>onError</code>来终止并处理错误。从上层看：</p>
<ul>
<li><code>execute</code>方法将会抛出异常</li>
<li><code>queue</code>方法将会返回一个失败状态的<code>Future</code>对象</li>
<li><code>observe()</code>和<code>toObservable()</code>方法都会返回上述的<code>Observable</code>对象</li>
</ul>
<h1 id="HystrixCircuitBreaker源码分析">HystrixCircuitBreaker源码分析</h1><p>Hystrix中的Circuit Breaker的实现比较明了。整个<code>HystrixCircuitBreaker</code>接口一共有三个方法和三个静态类：</p>
<p><img src="/assets/blog-img-k1/hystrix-circuit-breaker-class-structure.png" alt="Class Hierarchy of HystrixCircuitBreaker"></p>
<p>其中<code>allowRequest()</code>方法表示是否允许指令执行，<code>isOpen()</code>方法表示断路器是否为开启状态，<code>markSuccess()</code>用于将断路器关闭。</p>
<p><code>Factory</code>静态类相当于Circuit Breaker Factory，用于获取相应的<code>HystrixCircuitBreaker</code>。我们来看一下其实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="class"><span class="keyword">class</span> <span class="title">Factory</span> </span>&#123;</div><div class="line">    <span class="comment">// String is HystrixCommandKey.name() (we can't use HystrixCommandKey directly as we can't guarantee it implements hashcode/equals correctly)</span></div><div class="line">    <span class="keyword">private</span> <span class="keyword">static</span> ConcurrentHashMap&lt;String, HystrixCircuitBreaker&gt; circuitBreakersByCommand = <span class="keyword">new</span> ConcurrentHashMap&lt;String, HystrixCircuitBreaker&gt;();</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> HystrixCircuitBreaker <span class="title">getInstance</span><span class="params">(HystrixCommandKey key, HystrixCommandGroupKey group, HystrixCommandProperties properties, HystrixCommandMetrics metrics)</span> </span>&#123;</div><div class="line">        <span class="comment">// this should find it for all but the first time</span></div><div class="line">        HystrixCircuitBreaker previouslyCached = circuitBreakersByCommand.get(key.name());</div><div class="line">        <span class="keyword">if</span> (previouslyCached != <span class="keyword">null</span>) &#123;</div><div class="line">            <span class="keyword">return</span> previouslyCached;</div><div class="line">        &#125;</div><div class="line"></div><div class="line">        <span class="comment">// if we get here this is the first time so we need to initialize</span></div><div class="line"></div><div class="line">        <span class="comment">// Create and add to the map ... use putIfAbsent to atomically handle the possible race-condition of</span></div><div class="line">        <span class="comment">// 2 threads hitting this point at the same time and let ConcurrentHashMap provide us our thread-safety</span></div><div class="line">        <span class="comment">// If 2 threads hit here only one will get added and the other will get a non-null response instead.</span></div><div class="line">        HystrixCircuitBreaker cbForCommand = circuitBreakersByCommand.putIfAbsent(key.name(), <span class="keyword">new</span> HystrixCircuitBreakerImpl(key, group, properties, metrics));</div><div class="line">        <span class="keyword">if</span> (cbForCommand == <span class="keyword">null</span>) &#123;</div><div class="line">            <span class="comment">// this means the putIfAbsent step just created a new one so let's retrieve and return it</span></div><div class="line">            <span class="keyword">return</span> circuitBreakersByCommand.get(key.name());</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">            <span class="comment">// this means a race occurred and while attempting to 'put' another one got there before</span></div><div class="line">            <span class="comment">// and we instead retrieved it and will now return it</span></div><div class="line">            <span class="keyword">return</span> cbForCommand;</div><div class="line">        &#125;</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> HystrixCircuitBreaker <span class="title">getInstance</span><span class="params">(HystrixCommandKey key)</span> </span>&#123;</div><div class="line">        <span class="keyword">return</span> circuitBreakersByCommand.get(key.name());</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="comment">/* package */</span><span class="function"><span class="keyword">static</span> <span class="keyword">void</span> <span class="title">reset</span><span class="params">()</span> </span>&#123;</div><div class="line">        circuitBreakersByCommand.clear();</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>Hystrix在<code>Factory</code>类中维护了一个<code>ConcurrentHashMap</code>用于存储与每一个<code>HystrixCommandKey</code>相对应的<code>HystrixCircuitBreaker</code>。每当我们通过<code>getInstance</code>方法从中获取<code>HystrixCircuitBreaker</code>的时候，Hystrix首先会检查<code>ConcurrentHashMap</code>中有没有对应的缓存的断路器，如果有的话直接返回。如果没有的话就会新创建一个<code>HystrixCircuitBreaker</code>实例，将其添加到缓存中并且返回。</p>
<p><code>HystrixCircuitBreakerImpl</code>静态类是<code>HystrixCircuitBreaker</code>接口的实现。我们可以看到<code>HystrixCircuitBreakerImpl</code>类中有四个成员变量。其中<code>properties</code>是对应<code>HystrixCommand</code>的属性类，<code>metrics</code>是对应<code>HystrixCommand</code>的度量数据类。由于会工作在并发环境下，我们用一个<code>AtomicBoolean</code>类型的变量<code>circuitOpen</code>来代表断路器的状态（默认是<code>false</code>代表关闭，这里没有特意实现Half-Open这个状态），并用一个<code>AtomicLong</code>类型的变量<code>circuitOpenedOrLastTestedTime</code>记录着断路恢复计时器的初始时间，用于Open状态向Close状态的转换。</p>
<p>我们首先来看一下<code>isOpen</code>方法的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">isOpen</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">if</span> (circuitOpen.get()) &#123;</div><div class="line">        <span class="comment">// if we're open we immediately return true and don't bother attempting to 'close' ourself as that is left to allowSingleTest and a subsequent successful test to close</span></div><div class="line">        <span class="keyword">return</span> <span class="keyword">true</span>;</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="comment">// we're closed, so let's see if errors have made us so we should trip the circuit open</span></div><div class="line">    HealthCounts health = metrics.getHealthCounts();</div><div class="line"></div><div class="line">    <span class="comment">// check if we are past the statisticalWindowVolumeThreshold</span></div><div class="line">    <span class="keyword">if</span> (health.getTotalRequests() &lt; properties.circuitBreakerRequestVolumeThreshold().get()) &#123;</div><div class="line">        <span class="comment">// we are not past the minimum volume threshold for the statisticalWindow so we'll return false immediately and not calculate anything</span></div><div class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="keyword">if</span> (health.getErrorPercentage() &lt; properties.circuitBreakerErrorThresholdPercentage().get()) &#123;</div><div class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">    &#125; <span class="keyword">else</span> &#123;</div><div class="line">        <span class="comment">// our failure rate is too high, trip the circuit</span></div><div class="line">        <span class="keyword">if</span> (circuitOpen.compareAndSet(<span class="keyword">false</span>, <span class="keyword">true</span>)) &#123;</div><div class="line">            <span class="comment">// if the previousValue was false then we want to set the currentTime</span></div><div class="line">            circuitOpenedOrLastTestedTime.set(System.currentTimeMillis());</div><div class="line">            <span class="keyword">return</span> <span class="keyword">true</span>;</div><div class="line">        &#125; <span class="keyword">else</span> &#123;</div><div class="line">            <span class="comment">// How could previousValue be true? If another thread was going through this code at the same time a race-condition could have</span></div><div class="line">            <span class="comment">// caused another thread to set it to true already even though we were in the process of doing the same</span></div><div class="line">            <span class="comment">// In this case, we know the circuit is open, so let the other thread set the currentTime and report back that the circuit is open</span></div><div class="line">            <span class="keyword">return</span> <span class="keyword">true</span>;</div><div class="line">        &#125;</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先通过<code>circuitOpen.get()</code>获取断路器的状态，如果是开启状态(<code>true</code>)则返回<code>true</code>。否则，Hystrix会从Metrics数据中获取<code>HealthCounts</code>对象，然后检查对应的请求总数(<code>totalCount</code>)是否小于属性中的请求容量阈值(<code>circuitBreakerRequestVolumeThreshold</code>)，如果是的话表示断路器可以保持关闭状态，返回<code>false</code>。如果不满足请求总数条件，就再检查错误比率(<code>errorPercentage</code>)是否小于属性中的错误百分比阈值(<code>circuitBreakerErrorThresholdPercentage</code>，默认 <strong>50</strong>)，如果是的话表示断路器可以保持关闭状态，返回 <code>false</code>；如果超过阈值，Hystrix会判定服务的某些地方出现了问题，因此通过CAS操作将断路器设为开启状态，并记录此时的系统时间作为定时器初始时间，最后返回 <code>true</code>。</p>
<p>我们再来看一下判断Open状态下计时器的实现方法<code>allowSingleTest</code>:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">allowSingleTest</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">long</span> timeCircuitOpenedOrWasLastTested = circuitOpenedOrLastTestedTime.get();</div><div class="line">    <span class="comment">// 1) if the circuit is open</span></div><div class="line">    <span class="comment">// 2) and it's been longer than 'sleepWindow' since we opened the circuit</span></div><div class="line">    <span class="keyword">if</span> (circuitOpen.get() &amp;&amp; System.currentTimeMillis() &gt; timeCircuitOpenedOrWasLastTested + properties.circuitBreakerSleepWindowInMilliseconds().get()) &#123;</div><div class="line">        <span class="comment">// We push the 'circuitOpenedTime' ahead by 'sleepWindow' since we have allowed one request to try.</span></div><div class="line">        <span class="comment">// If it succeeds the circuit will be closed, otherwise another singleTest will be allowed at the end of the 'sleepWindow'.</span></div><div class="line">        <span class="keyword">if</span> (circuitOpenedOrLastTestedTime.compareAndSet(timeCircuitOpenedOrWasLastTested, System.currentTimeMillis())) &#123;</div><div class="line">            <span class="comment">// if this returns true that means we set the time so we'll return true to allow the singleTest</span></div><div class="line">            <span class="comment">// if it returned false it means another thread raced us and allowed the singleTest before we did</span></div><div class="line">            <span class="keyword">return</span> <span class="keyword">true</span>;</div><div class="line">        &#125;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>首先获取断路恢复计时器记录的初始时间<code>circuitOpenedOrLastTestedTime</code>，然后判断以下两个条件是否同时满足：</p>
<ul>
<li>断路器的状态为开启状态(<code>circuitOpen.get() == true</code>)</li>
<li>当前时间与计时器初始时间之差大于计时器阈值<code>circuitBreakerSleepWindowInMilliseconds</code>(默认为 <strong>5</strong> 秒)</li>
</ul>
<p>如果同时满足的话，表示可以从<code>Open</code>状态向<code>Close</code>状态转换。Hystrix会通过CAS操作将<code>circuitOpenedOrLastTestedTime</code>设为当前时间，并返回<code>true</code>。如果不同时满足，返回<code>false</code>，代表断路器关闭或者计时器时间未到。</p>
<p>有了这个函数以后，我们再来看一下<code>allowRequest</code>的实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="meta">@Override</span></div><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">allowRequest</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">if</span> (properties.circuitBreakerForceOpen().get()) &#123;</div><div class="line">        <span class="comment">// properties have asked us to force the circuit open so we will allow NO requests</span></div><div class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">if</span> (properties.circuitBreakerForceClosed().get()) &#123;</div><div class="line">        <span class="comment">// we still want to allow isOpen() to perform it's calculations so we simulate normal behavior</span></div><div class="line">        isOpen();</div><div class="line">        <span class="comment">// properties have asked us to ignore errors so we will ignore the results of isOpen and just allow all traffic through</span></div><div class="line">        <span class="keyword">return</span> <span class="keyword">true</span>;</div><div class="line">    &#125;</div><div class="line">    <span class="keyword">return</span> !isOpen() || allowSingleTest();</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>非常直观。首先先读取属性中的强制设定值（可以强制设定状态），如果没有设定的话，就判断断路器是否关闭或者断路恢复计时器是否到达时间，只要满足其中一个条件就返回<code>true</code>，即允许执行操作指令。</p>
<p>最后就是<code>markSuccess</code>方法了，它用于关闭断路器并重置统计数据。代码非常直观，就不多说了：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">markSuccess</span><span class="params">()</span> </span>&#123;</div><div class="line">    <span class="keyword">if</span> (circuitOpen.get()) &#123;</div><div class="line">        <span class="keyword">if</span> (circuitOpen.compareAndSet(<span class="keyword">true</span>, <span class="keyword">false</span>)) &#123;</div><div class="line">            metrics.resetStream();</div><div class="line">        &#125;</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>Hystrix的Circuit Breaker可以用以下的图来总结：</p>
<p><img src="/assets/blog-img-k1/hystrix-circuit-breaker-1280.png" alt="Hystrix Circuit Breaker"></p>
<p>至于Hystrix在底层执行Command时是如何利用<code>HystrixCircuitBreaker</code>的，可以看<code>AbstractCommand</code>类中<code>toObservable</code>方法和<code>getRunObservableDecoratedForMetricsAndErrorHandling</code>方法的源码，后边再总结。</p>
<hr>
<h1 id="Reference">Reference</h1><ul>
<li><a href="https://github.com/Netflix/Hystrix/wiki/How-it-Works" target="_blank" rel="external">How it Works - Netflix/Hystrix - Wiki</a></li>
</ul>
]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;2017-12-04：待重新总结。。。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;这篇文章里我们来总结一下 Netflix Hystrix 的工作流程（版本为 1.4.x）。这是官方提供的流程图（来自 &lt;a href=&quot;https://github
    
    </summary>
    
      <category term="Microservice" scheme="http://www.sczyh30.com/categories/Microservice/"/>
    
    
      <category term="微服务" scheme="http://www.sczyh30.com/tags/%E5%BE%AE%E6%9C%8D%E5%8A%A1/"/>
    
      <category term="分布式系统" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="Netflix Hystrix" scheme="http://www.sczyh30.com/tags/Netflix-Hystrix/"/>
    
      <category term="高可用架构" scheme="http://www.sczyh30.com/tags/%E9%AB%98%E5%8F%AF%E7%94%A8%E6%9E%B6%E6%9E%84/"/>
    
      <category term="限流降级" scheme="http://www.sczyh30.com/tags/%E9%99%90%E6%B5%81%E9%99%8D%E7%BA%A7/"/>
    
  </entry>
  
  <entry>
    <title>并发编程算法 | Treiber Stack 实现 lock-free stack</title>
    <link href="http://www.sczyh30.com/posts/Concurrency/concurrency-treiber-stack/"/>
    <id>http://www.sczyh30.com/posts/Concurrency/concurrency-treiber-stack/</id>
    <published>2016-05-21T16:00:00.000Z</published>
    <updated>2018-09-23T13:02:46.000Z</updated>
    
    <content type="html"><![CDATA[<p>多线程环境下各种数据结构的实现有了很大的变化，每当我们更新某个数据的时候，我们都要考虑其它线程是否对其进行了修改。最简单的一种方法就是加锁，不过加锁会导致性能低下，而且可能阻塞其他线程。因此，我们引入了非阻塞 (non-blocking) 的算法 —— 通过 CAS 操作保证操作的原子性，同时我们还引入了 <em>lock-free</em> 的概念，它指的是一个线程出现问题（如阻塞，失败）但不影响其他线程（从总体看程序仍然是在运行的）。这里就来看一下 Non-blocking stack 的一个实现 —— <strong>Treiber Stack</strong>。</p>
<h1 id="Treiber_Stack">Treiber Stack</h1><p>这里给出的是 Treiber Stack 的一个简化版的 Java 实现：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">import</span> java.util.concurrent.atomic.AtomicReference;</div><div class="line"></div><div class="line"><span class="comment">/**</span></div><div class="line"> * Concurrent stack implementation</div><div class="line"> * Treiber's Algorithm</div><div class="line"> */</div><div class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">ConcurrentStack</span>&lt;<span class="title">E</span>&gt; </span>&#123;</div><div class="line">    <span class="keyword">private</span> AtomicReference&lt;Node&lt;E&gt;&gt; top = <span class="keyword">new</span> AtomicReference&lt;&gt;();</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">push</span><span class="params">(E elem)</span> </span>&#123;</div><div class="line">        Node&lt;E&gt; newHead = <span class="keyword">new</span> Node&lt;&gt;(elem);</div><div class="line">        Node&lt;E&gt; oldHead;</div><div class="line">        <span class="keyword">do</span> &#123;</div><div class="line">            oldHead = top.get();</div><div class="line">            newHead.next = oldHead;</div><div class="line">        &#125; <span class="keyword">while</span> (!top.compareAndSet(oldHead, newHead));</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="function"><span class="keyword">public</span> E <span class="title">pop</span><span class="params">()</span> </span>&#123;</div><div class="line">        Node&lt;E&gt; oldHead;</div><div class="line">        Node&lt;E&gt; newHead;</div><div class="line">        <span class="keyword">do</span> &#123;</div><div class="line">            oldHead = top.get();</div><div class="line">            <span class="keyword">if</span> (oldHead == <span class="keyword">null</span>)</div><div class="line">                <span class="keyword">return</span> <span class="keyword">null</span>;</div><div class="line">            newHead = oldHead.next;</div><div class="line">        &#125; <span class="keyword">while</span> (!top.compareAndSet(oldHead, newHead));</div><div class="line">        <span class="keyword">return</span> oldHead.item;</div><div class="line">    &#125;</div><div class="line"></div><div class="line">    <span class="keyword">private</span> <span class="keyword">static</span> <span class="class"><span class="keyword">class</span> <span class="title">Node</span>&lt;<span class="title">E</span>&gt; </span>&#123;</div><div class="line">        <span class="keyword">public</span> <span class="keyword">final</span> E item;</div><div class="line">        <span class="keyword">public</span> Node&lt;E&gt; next;</div><div class="line"></div><div class="line">        <span class="function"><span class="keyword">public</span> <span class="title">Node</span><span class="params">(E item)</span> </span>&#123;</div><div class="line">            <span class="keyword">this</span>.item = item;</div><div class="line">        &#125;</div><div class="line">    &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>我们使用了 <code>AtomicReference</code> 来实现 Treiber Stack。每当我们 <code>push</code> 进去一个元素的时候，我们首先根据要添加的元素创建一个 <code>Node</code>，然后获取原栈顶结点，并将新结点的下一个结点指向原栈顶结点。此时我们使用 CAS 操作来更改栈顶结点，如果此时的栈顶和之前的相同，代表 CAS 操作成功，那么就把新插入的元素设为栈顶；如果此时的栈顶和之前的不同（即其他线程改变了栈顶结点），CAS 操作失败，那么需要重复上述操作（更新当前的栈顶元素并且重设 next），直到成功。<code>pop</code> 操作的原理也相似。</p>
<hr>
<h1 id="References">References</h1><ul>
<li><em>Java Concurrency In Practice</em></li>
</ul>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;多线程环境下各种数据结构的实现有了很大的变化，每当我们更新某个数据的时候，我们都要考虑其它线程是否对其进行了修改。最简单的一种方法就是加锁，不过加锁会导致性能低下，而且可能阻塞其他线程。因此，我们引入了非阻塞 (non-blocking) 的算法 —— 通过 CAS 操作保
    
    </summary>
    
      <category term="Concurrency" scheme="http://www.sczyh30.com/categories/Concurrency/"/>
    
    
      <category term="并发" scheme="http://www.sczyh30.com/tags/%E5%B9%B6%E5%8F%91/"/>
    
      <category term="Algorithm" scheme="http://www.sczyh30.com/tags/Algorithm/"/>
    
      <category term="多线程" scheme="http://www.sczyh30.com/tags/%E5%A4%9A%E7%BA%BF%E7%A8%8B/"/>
    
  </entry>
  
  <entry>
    <title>分布式系统基础总结</title>
    <link href="http://www.sczyh30.com/posts/Distributed-System/distributed-system-base/"/>
    <id>http://www.sczyh30.com/posts/Distributed-System/distributed-system-base/</id>
    <published>2016-05-21T16:00:00.000Z</published>
    <updated>2017-02-24T06:32:44.000Z</updated>
    
    <content type="html"><![CDATA[<p>从大一就一直在搞分布式系统，Hadoop, Spark, Kafka, ZooKeeper之类的都玩过，然而以前只是简单地用用各个开源组件实现，并没有系统地学习其中的原理和算法。最近在跟着MIT 6.824课程学习分布式系统的各种理论原理，这里就来简单总结下分布式系统中的一些基础内容吧~</p>
<h1 id="CAP_Theory">CAP Theory</h1><p>CAP Theory阐述了分布式系统中的一个事实：一致性(Consistency)、可用性(Availability)和分区容错性(Partition Tolerance)不能同时保证。三个只能选择两个</p>
<p><img src="http://book.mixu.net/distsys/images/CAP.png" alt="CAP Theory"></p>
<p>假设有两台机器A、B，两者之间互相同步保持数据的一致性。现在B由于网络原因不能与A通信(Network Partition)，假设某个client向A写入数据，现在有两种选择：</p>
<ul>
<li>A拒绝写入，这样能保证与B的一致性，但是牺牲了可用性</li>
<li>A允许写入，但是这样就不能保证与B的一致性了</li>
</ul>
<p>Network Partition是必然的，网络非常可能出现问题（断线、超时），因此CAP理论一般只能AP或CP，而CA一般较难实现。</p>
<ul>
<li>CP: 要实现一致性，则需要一定的一致性算法，一般是基于多数派表决的，如Paxos和Raft</li>
<li>AP: 要实现可用性，则要有一定的策略决议到底用哪个数据，并且数据一般要进行冗余备份(replication)</li>
</ul>
<p>当然，在上面的例子中，A可以先允许写入，等B的网络恢复以后再同步至B（根据CAP原理这样不能保证强一致性了，但是可以考虑实现最终一致性）。</p>
<h1 id="一致性哈希">一致性哈希</h1><p>分布式Key-Value Store中的key映射问题。</p>
<ul>
<li>传统<code>hash(x) % N</code>算法的弊端：不利于架构的伸缩性</li>
<li>一致性哈希将哈希值映射到一个哈希环上，然后将数据进行哈希处理后映射到哈希环上，再把节点进行哈希处理映射到哈希环上，数据将选择最近的节点存储</li>
<li>伸缩性：节点删除时，原有的数据将会就近迁移，其他数据不用迁移；节点增加时也相似</li>
<li>保证负载均衡：虚拟节点</li>
</ul>
<h1 id="拜占庭将军问题">拜占庭将军问题</h1><p>最复杂的情况：自己发的包被截；对方发的包自己收不到；内部有节点捣乱，造成不一致。</p>
<h1 id="FLP_Impossibility">FLP Impossibility</h1><p><em>Impossibility of Distributed Consensus with One Faulty Process</em> 这篇论文提到：</p>
<blockquote>
<p>No completely asynchronous consensus protocol can tolerate even a single unannounced process death.</p>
</blockquote>
<p>假设节点只有崩溃这一种异常行为，网络是可靠的，并且不考虑异步通信下的时序差异。FLP Impossibility指出在异步网络环境中只要有一个故障节点, 任何Consensus算法都无法保证行为正确。</p>
<h1 id="Lease机制">Lease机制</h1><p>Lease（租约）机制应用非常广泛：</p>
<ul>
<li>可用作授权来进行同步等操作(如Append)</li>
<li>可用作读锁/写锁（分布式锁）</li>
</ul>
<p>租约的一个关键点就是有效期，过了有效期可以续约。如果不可用就收回租约，给另一台服务器权限。</p>
<p>实际应用：</p>
<ul>
<li>GFS: Master grant to ChunckServer</li>
</ul>
<blockquote>
<p>思考：<strong>Lease == Lock?</strong></p>
</blockquote>
<h1 id="Quorum机制">Quorum机制</h1><p>多数表决机制在分布式系统中通常有两个应用场景：</p>
<ol>
<li>Leader Election</li>
<li>Replication (NWR机制)</li>
</ol>
<p>理论基础：鸽巢原理</p>
<h1 id="Consensus问题">Consensus问题</h1><h2 id="Consensus条件">Consensus条件</h2><ul>
<li>Termination: 最终必须决议出结果</li>
<li>Validity:</li>
<li>Integrity</li>
<li>Agreement</li>
</ul>
<h2 id="2PC/3PC">2PC/3PC</h2><p>2PC在proposer和某个voter都挂掉的时候会阻塞（原因：别的节点没有对应voter的消息，只能阻塞等待此voter恢复）</p>
<p>3PC添加了一个  <em>prepare-commit</em> 阶段用于准备提交工作，这里面可以实现事务的回滚。</p>
<p>缺点：效率貌似很低。。。分布式事务用2PC会特别蛋疼</p>
<h1 id="Paxos">Paxos</h1><p>推演：</p>
<ul>
<li>First Accept/Last Accept都不可以（结合时序图）</li>
<li>一个阶段不行，自然想到两个阶段：发起决议以及提交决议</li>
</ul>
<ul>
<li>One Proposer -&gt; One Acceptor 挂了就gg了</li>
<li>One Proposer -&gt; Many Acceptors （规则：先到先投）</li>
<li>Many Proposers -&gt; Many Acceptors (很容易乱。。。)</li>
</ul>
<p>Paxos引入了Log ID (num, value)，共有三个角色，两个阶段。</p>
<ul>
<li>Proposer: 决议发起者，需要等待多数派表决</li>
<li>Acceptor: 决议投票者，对收到的Propose进行表决并返回</li>
<li>Learner: 最水的角色，学习到投票的结果即可</li>
</ul>
<blockquote>
<p>分布式一致性算法(Paxos, Raft, Chubby, Zab)待详细总结。。。</p>
</blockquote>
<h1 id="时序问题">时序问题</h1><h2 id="Lamport_Timestamp">Lamport Timestamp</h2><p>一般我们不关心分布式系统中某个过程的绝对时间，而只关注两个事件之间的相对时间。<br>在一个系统的事件集合E上定义一种偏序关系<code>-&gt;</code>，满足：</p>
<ul>
<li>如果a与b发生在同一个进程中，且a先于b发生，则有<code>a -&gt; b</code></li>
<li>进程间通信，a代表进程Pi发出消息m，b代表另一个进程Pj接收消息m，则有<code>a -&gt; b</code></li>
<li>传递性：若<code>a -&gt; b, b -&gt; c</code>，则<code>a -&gt; c</code></li>
</ul>
<p>定义并发：<code>a -&gt; b</code>与<code>b -&gt; a</code>均不成立则为并发情况</p>
<p>引入Lamport逻辑时钟。一个时钟本质上是一个事件到实数的映射（假设时间是连续的）。对于每一个进程Pi，都有其对应的时钟Ci。</p>
<p>分布式系统中的全局信息实际上是对各个实体信息的叠加（Q：重合怎么办？）</p>
<p>可以看到Lamport Timestamp必须要求两个事件有先后顺序关系，因而在时序图上不好表示concurrent。由此引入Vector Clock。</p>
<h2 id="Vector_Clock">Vector Clock</h2><p>Vector Clock是对Lamport Timestamp的演进。它不仅保存了自身的timestamp，而且还保留了根节点的timestamp。</p>
<p>Vector Clock(Version Vector)只能用于发现数据冲突，但是想要解决数据冲突还要留给用户去定夺（就好比git commit出现conflicts，需要手工解决一样），当然也可以设置某种策略来直接解决冲突（保留最新或集群内多数表决）。</p>
<p>结合时序图理解会更好(图来自维基百科)：</p>
<p><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/5/55/Vector_Clock.svg/725px-Vector_Clock.svg.png" alt=""></p>
<p>可能出现的问题：Vector Clock过多。解决方案：剪枝（如果超过某个阈值就把最初的那个给扔掉；要是现在还依赖最初的那个clock的话可能就会造成一些问题（思考：如何解决？）</p>
<p>对应论文：<em>Dynamo: Amazon’s Highly Available Key-value Store, Section 4.4</em>.</p>
<hr>
<h1 id="References">References</h1><ul>
<li><a href="http://book.mixu.net/distsys/index.html" target="_blank" rel="external">Distributed systems for fun and profit</a></li>
<li><a href="http://research.microsoft.com/users/lamport/pubs/time-clocks.pdf" target="_blank" rel="external">Leslie Lamport. Time, Clocks and Ordering of Events in a Distributed System.</a></li>
<li><a href="https://groups.csail.mit.edu/tds/papers/Lynch/jacm85.pdf" target="_blank" rel="external">Michael J. Fisher, et al. Impossibility of Distributed Consensus with One Faulty Process.</a></li>
<li><a href="http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf" target="_blank" rel="external">Giuseppe DeCandia, et al. Dynamo: Amazon’s Highly Available Key-value Store.</a></li>
</ul>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;从大一就一直在搞分布式系统，Hadoop, Spark, Kafka, ZooKeeper之类的都玩过，然而以前只是简单地用用各个开源组件实现，并没有系统地学习其中的原理和算法。最近在跟着MIT 6.824课程学习分布式系统的各种理论原理，这里就来简单总结下分布式系统中的一
    
    </summary>
    
      <category term="Distributed System" scheme="http://www.sczyh30.com/categories/Distributed-System/"/>
    
    
      <category term="分布式系统" scheme="http://www.sczyh30.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/"/>
    
      <category term="CAP" scheme="http://www.sczyh30.com/tags/CAP/"/>
    
  </entry>
  
  <entry>
    <title>使用 Docker Compose 编排容器集群</title>
    <link href="http://www.sczyh30.com/posts/Architecture/service-with-docker-compose/"/>
    <id>http://www.sczyh30.com/posts/Architecture/service-with-docker-compose/</id>
    <published>2016-05-19T16:00:00.000Z</published>
    <updated>2018-09-23T14:15:44.000Z</updated>
    
    <content type="html"><![CDATA[<p><a href="https://docs.docker.com/compose/" target="_blank" rel="external">Docker Compose</a>可以让我们方便、快速地编排容器集群，有利于分布式架构的实现。</p>
<p>假设我们有一个应用以MySQL作为数据存储，如果没有编排工具的话，我们在构建此应用的Docker镜像时必须将MySQL一同打包进镜像中，这样不仅会使镜像体积臃肿，而且不利于分布式架构的实现（假如要做读写分离、主从复制之类的）。而有了Docker Compose，我们就可以创建两个镜像：单独的应用镜像和MySQL镜像。在运行时，分别创建两个容器，并且将两个容器链接(link)在一起，使它们之间可以按照配置相互通信。这样就将我们的单体应用拆分成了多个组件构成的应用（其实这就是微服务的思想），从而更有利于服务间的解耦以及分布式架构的实现。</p>
<p>下面举一个例子，完整实现可见<a href="https://github.com/sczyh30/vertx-blueprint-todo-backend" target="_blank" rel="external">vertx-blueprint-todo-backend | GitHub</a>。现有一服务<code>Vert.x Blueprint Todo Backend</code>已打包成jar包，该服务以Redis作为数据存储。该服务以及Redis监听的地址和端口通过JSON配置文件来提供。我们可以设计两个镜像：服务镜像（通过<code>Dockerfile</code>构建）以及官方Redis镜像，运行时分别创建一个容器实例，然后通过Docker Compose将两个容器组合起来。首先来看一下我们的<code>Dockerfile</code>：</p>
<figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">FROM</span> java:<span class="number">8</span>-jre</div><div class="line"></div><div class="line"><span class="keyword">ENV</span> VERTICLE_FILE build/libs/vertx-blueprint-todo-backend-fat.jar</div><div class="line"></div><div class="line"><span class="keyword">ENV</span> VERTICLE_HOME /usr/verticles</div><div class="line"></div><div class="line"><span class="keyword">EXPOSE</span> <span class="number">8082</span></div><div class="line"></div><div class="line"><span class="keyword">COPY</span><span class="bash"> <span class="variable">$VERTICLE_FILE</span> <span class="variable">$VERTICLE_HOME</span>/</span></div><div class="line"><span class="keyword">COPY</span><span class="bash"> config/config_docker.json <span class="variable">$VERTICLE_HOME</span>/</span></div><div class="line"></div><div class="line"><span class="keyword">WORKDIR</span><span class="bash"> <span class="variable">$VERTICLE_HOME</span></span></div><div class="line"><span class="keyword">ENTRYPOINT</span><span class="bash"> [<span class="string">"sh"</span>, <span class="string">"-c"</span>]</span></div><div class="line"><span class="keyword">CMD</span><span class="bash"> [<span class="string">"java -jar vertx-blueprint-todo-backend-fat.jar -conf config_docker.json"</span>]</span></div></pre></td></tr></table></figure>
<p>服务容器运行时对外暴露8082端口。再看一下服务配置文件：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line">&#123;</div><div class="line">  <span class="attr">"service.type"</span>: <span class="string">"redis"</span>,</div><div class="line">  <span class="attr">"http.port"</span>: <span class="number">8082</span>,</div><div class="line">  <span class="attr">"http.address"</span>: <span class="string">"0.0.0.0"</span>,</div><div class="line">  <span class="attr">"redis.host"</span>: <span class="string">"redis"</span>,</div><div class="line">  <span class="attr">"redis.port"</span>: <span class="number">6379</span></div><div class="line">&#125;</div></pre></td></tr></table></figure>
<p>注意我们将Redis的host设为<code>redis</code>，这个<code>redis</code>是对应的访问路径，后面会提到。下面来看Docker Compose的配置文件<code>docker-compose.yml</code>：</p>
<figure class="highlight yaml"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line"><span class="attr">version:</span> <span class="string">"2"</span></div><div class="line"><span class="attr">services:</span></div><div class="line"><span class="attr">  redis:</span></div><div class="line"><span class="attr">    container_name:</span> vertx-todo-backend_redis</div><div class="line"><span class="attr">    image:</span> redis:latest</div><div class="line"><span class="attr">    expose:</span></div><div class="line"><span class="bullet">      -</span> <span class="string">"6379"</span></div><div class="line"></div><div class="line"><span class="attr">  vertx-todo-backend:</span></div><div class="line"><span class="attr">    depends_on:</span></div><div class="line"><span class="bullet">      -</span> redis</div><div class="line"><span class="attr">    container_name:</span> vertx-todo-backend_service</div><div class="line"><span class="attr">    build:</span> .</div><div class="line"><span class="attr">    links:</span></div><div class="line"><span class="bullet">      -</span> redis</div><div class="line"><span class="attr">    ports:</span></div><div class="line"><span class="bullet">      -</span> <span class="string">"8082:8082"</span></div></pre></td></tr></table></figure>
<p>其中<code>version: &quot;2&quot;</code>代表对应镜像版本，最新的需要Docker 1.10.0支持。在<code>services</code>中，我们定义了两个<code>service</code>：<code>redis</code>和<code>vertx-todo-backend</code>。</p>
<p>先来看<code>redis</code>中的配置。<code>container_name</code>代表容器名称，<code>image</code>代表对应的镜像，<code>expose</code>代表在集群内暴露的端口号（不对外暴露）。其它容器通过服务名<code>redis</code>访问此镜像。</p>
<p>再来看<code>vertx-todo-backend</code>。我们的服务需要依赖Redis，因此容器的启动顺序应该是<code>redis -&gt; vertx-todo-backend</code>，因此我们配置了<code>depends_on</code>选项，此选项下的所有容器都将会在本容器启动之前启动（注意只是启动，并不是其它容器初始化完成后本容器才启动。如果需要等待其它容器初始化完毕，则需要另写脚本）。<code>build</code>对应着<code>Dockerfile</code>文件的路径，<code>links</code>代表链接的镜像，<code>ports</code>代表对外暴露的端口。</p>
<p>配置好以后，我们在目录下执行<code>docker-compose up --build</code>，一会就可以看到容器集群运行起来了，非常方便。</p>
<p><img src="/assets/blog-img-k1/vbptds-docker-compose-running.png" alt="Docker Compose"></p>
<p>更多的有关Docker Compose的信息，参考<a href="https://docs.docker.com/compose/" target="_blank" rel="external">官方文档</a>。</p>
]]></content>
    
    <summary type="html">
    
      &lt;p&gt;&lt;a href=&quot;https://docs.docker.com/compose/&quot; target=&quot;_blank&quot; rel=&quot;external&quot;&gt;Docker Compose&lt;/a&gt;可以让我们方便、快速地编排容器集群，有利于分布式架构的实现。&lt;/p&gt;
&lt;p&gt;假设我们有一个
    
    </summary>
    
      <category term="Architecture" scheme="http://www.sczyh30.com/categories/Architecture/"/>
    
    
      <category term="微服务" scheme="http://www.sczyh30.com/tags/%E5%BE%AE%E6%9C%8D%E5%8A%A1/"/>
    
      <category term="Docker" scheme="http://www.sczyh30.com/tags/Docker/"/>
    
      <category term="容器" scheme="http://www.sczyh30.com/tags/%E5%AE%B9%E5%99%A8/"/>
    
  </entry>
  
</feed>
