Author Archives: admin

说说《大圣归来》

其实本来我是不打算看这部电影的,刚好公司组织一起去看,原价35的3D票只要10块钱,算是个福利,似乎网上一片称赞,索性就看一看。我最近刚把 GTA 5 打通,一篇很长的评论还没写完,就先来说说这个《大圣归来》。

首先得感谢这电影票只花了10块钱,要是35买的看完我肯定觉得不值。接下来一样样的评。

画面:很好。这部电影在画面上的突破,相对于其它国产动画还是很明显的。不夸张的说,这样的画面素质,放到迪士尼的那一堆3D动画片里也是不算差的。毛发的细节,光照,物理效果,都很好。可惜的是,画面好也就止步于此了,这么好的画面没有用来营造气氛渲染情感,妖洞没有妖气,那个关大圣的水晶宫其实很漂亮可以多给点特写但是稍微摆弄了两下就过去了,没有物尽其用。

剧情:弱爆了。一个好的剧情,要么是讲一个大家都没听过的故事,要么是讲一个大家以为都听过实际上走向出乎意料的故事。前者是盗梦空间,后者是冰雪奇缘。大圣的故事,看到开头猜到结尾,结果导演连个结尾都不让我们看,连个完整的故事都不讲完故意吊着你,不想多说了。

人物:既然片名叫大圣归来,那么我想主角肯定是大圣吧,其它什么江流儿老师傅猪八戒应该都是配角才对吧?可是塑造出来了一个什么样的大圣形象呢?恕我眼拙看不出来,因为封印没解掉垂头丧气,因为看到江流儿被妖怪打而变身奥特曼,体现出大圣是一个爱护儿童的好猴子?可是论爱护儿童,我觉得师傅才是真·爱啊,人家没有特技都敢入妖穴,比大圣这种有特技的高到不知道哪里去了。

音乐:那个汪峰的歌一出来就彻底出戏了。现在抓妖精、救女孩的事情一个没做出这么燃的BGM是要怎样?好在其它的配乐分寸把握的都还好,至少不是负分。

总体而言,个人觉得这种某个方面有明显瑕疵的电影评分在7到8之间算是一个正常的评价。大圣归来,并没有好到值得让你专门抽两个小时的时间,认认真真欣赏的水平;如果大圣归来是迪士尼出的,我会毫不客气的把他划到流水线片、爆米花片的类别里去。

一个由于Nginx配置不当导致的启动失败: Stopping System V runlevel compatibility

最近在做Android Push系统的服务器端, 要用到1.9版Nginx引入的TCP代理功能, 由于Nginx默认的连接数太少, 我就按照之前改内核参数的习惯, 直接大手一挥直接把连接数加到了1000W. reload配置之后, 我的机器死掉了.

我当时根本没想到是Nginx的原因, 下意识的认为是我用的那个MQTT的库一定是泄露内存了, 然后果断重启机器.

然后就起不来了, 在Ubuntu的启动界面一直转啊转. 再次重启, 进 Recovery Mode 打日志, 发现卡在 Stopping System V runlevel compatibility [OK] 这里.

网上几乎一边倒的认为是 NVidia 的显卡驱动问题, 虽然我觉得不太可能是这个原因不过网上都这么说, 那就卸了吧.

卸了之后还是进不去系统, 而且还是卡在老地方! 于是开始在笔记本上查资料, 机器就放在那里没管. 过了几分钟之后, 我瞟了一眼机器, 出了一行 log, Out of Memory Error, Kill Nginx. 这时候我才意识到到是不是和我之前改了 Nginx 配置有关系. 很快我就确定是 Nginx 的问题, 因为 Nginx 用的是连接池, 即使没有连接也会预先创建好一定数量的备用, 我的机器大约8G内存, 之前的测试中大约能抗住80W左右的连接, 1000W的连接池必定导致OOM, 然后Nginx就吃光了所有的内存, 强迫操作系统不断进行垃圾回收, 导致启动卡死.

接下来就很简单了, 把Nginx的连接数改回去, 再把显卡驱动装上, 成功进入系统, 然后再给Nginx设置一个合适的连接池大小, 继续进行试验.

Closures in different languages

In most scripting languages, there are first-class functions. In short, first-class functions refer to functions that can serve as call arguments, work in expressions and be assigned to variables.

So what is a closure? A closure is a function that brings context information with it. Among all the languages, JavaScript is probably the language where closure is mostly frequently used. In my opinion, the reason why closures are so widely used in Javascript lies in that Javascript does not have a mature OO system compared to other programming languages.

The output of this piece of Javascript code is “2” and “3”. As we can see, function a() returned another function. However, this function not only carries the information about itself, but also the context it lies in, i.e. the value of variable t.

In Apple’s programming language Swift, we have similar closures that act almost identical to Javascript’s.

The result is also the same as Javascript’s, “2” and “3”. What about Python?

There are a few notable differences here. First of all, the variable t defined in a() is not visible to b(), thus it must be passed through named argument. Secondly, the output is “2” and “2”. This is reasonable since the value of t is passed through an argument and changing the value of the argument will not affect its original value. However, python do support “real” closures. The trick is to change the t into a nonlocal variable.

Now we are finally there. Next up, Groovy.

Since in Groovy, named functions cannot be defined in another function, we can only use unnamed function to do this. The output is “2” and “3”.

In Java, functions are not first-class members, thus we will never have terms like closure. A workaround for this is to use anonymous class. Here is an example.

Rule No.1 for anonymous class is that you cannot change the value of the variables that exist in the stack context. In other words, the variables on the stack are all “final” to the inner class. If one want’s to change the value of it, it must be declared as a field of a Class, which will be stored in heap.

Improve ListView Performance on Android

The performance of ListView on Android is sometimes a disaster when it comes to very complex list. Things become more frustrating when you are working with other things with Android like network images and dynamic loading.  The best example of a complex ListView is the Facebook feed in the Android app. They posted an article to show how complex but smooth ListView can be achieved at the same time.

https://code.facebook.com/posts/879498888759525/fast-rendering-news-feed-on-android/

In short, they split each post in the feed into several parts: the header, the main body and the action panel. Then each part will be able to be reused when rendering the ListView. This is a very clever alternative solution to the problematic ListView performance.

However, for my case https://github.com/cfan8/TGFC things are more complicated since the content of a post does not have a fixed style. It may be pure text, or text with some decorations, or full of images without a single line of text. Moreover, the content of the post should be interactive, i.e., when you click on a link or an image, the app should respond with different actions with that click.

In a previous open source Android app that I contributed to, we tried at least two options.

  1. Use a ListView with WebView, i.e., each item in the ListView is a WebView. In this case, it is easy to interact with other parts of the app while at the same time achieving high dynamic usability. Everything works fine when it is with Android 4.2 or before. Performance becomes a really big issue when it comes to Android 4.4, in which Google made the webkit kernel more functional but also heavier. Creating a WebView becomes a really time intensive task which we cannot afford. Thus we got several other workarounds for this problem.
    1. Keep as many WebViews as possible in the memory as cache. In the case that a user scroll down and up a WebView, the WebViews that are cached in the memory can be used directly. To keep the memory use to an acceptable level, we can use Soft Reference to cache each object.
      This workaround does not work well since when you scroll down the WebView, you are still creating new WebViews and it will only work when you scroll up and down, which is actually not very useful.
    2. Reuse each WebView. This does improved the scrolling experience since we no longer create a lot of WebViews. Instead, we alter the content of each WebView. The experience is still a little bit laggy since rendering HTML also takes a lot of time.
      This workaround worked better than the 1st one but it brings another big issue, i.e., when reusing the WebView, the height of it will not change when the length of its content changes. In other words, we will see a lot of blanks in the ListView when the length of each content of WebView varies significantly.
  2. Use a ScrollView with WebViews and render a page of posts at once. This is very brute but surprisingly work! The disadvantage of this solution is, firstly it is very memory consuming since the whole page of posts live inside the main memory. Secondly the app may froze for a second or two while rendering the page, depending on how complex the page is. However, once the page is rendered, it becomes super smooth no matter how you scroll it!

When I was figuring out the solution for the new app TGFC, I was thinking about what kind of solution I should implement. However, I realized that a WebView may not be the only option for my scenario since actually I don’t really need all the features that a heavy WebView provides. I want to have my app to be able to show some different styles of text, several images and that’s all. I don’t need stuffs like z-index or absolute positioning. In my case, TextView can work perfectly to meet my demands.

I started with a ListView of TextViews. At first, everything works fine when there is only text content in the TextView. However, things become a little bit complicated when I introduced network images into my app. Inside the ImageGetter that I was using, I first download images asynchronously to the local cache. Later on I load those images into memory and show them on the screen. I see notable lag when using ListView as the outside container when it was loading images from the local cache, so I switched to ScrollView later and rendered the whole page of posts at once.

The only thing that we need to be careful about is the usage of images. Large images can consume a lot of memory and make the ScrollView really laggy. Remember to resize those images when loading them into the memory.

Right now I still have some issues with the interaction between my TextView and the other parts of the app. I don’t have time to fix those issues and see whether they come from the HTML that Jsoup generates or the way I use TextView for the time being. But I’m pretty confident that these problems are not unsolvable and actually I have got some ideas on how to handle them. For the time being, TextView + ScrollView may be the solution for extreme complex and dynamic ListView with good user experience with better memory performance than WebView + ScrollView if you do not want to parse HTML and analyze the content to distinguish text parts and image parts.

This article is written as a complement to my Zhihu answer. In my opinion, the reason why we are having so many problems with ListView is the problematic designing of ListView that comes out from Google. Here are my suggestions on how to improve the performance of ListView from the Android designing perspective.

  1. Prepare more Views before scrolling. Currently the ListView will only prepare one more View that are invisible to the user but I believe its not enough. The number of views to be rendered should be extended.
  2. Android should introduce a kind of @PausableTask that is run on the UI thread but pausable to let the UI thread draw things to be shown on the screen. We can only show the basic outlines of the items in the initialization of a View and then gradually fill it with detailed content using the intervals of UI refresh, just like the way that Facebook used in its webpage, filling the page with place holders and filling those holders with content later on.

 

Android Support Library v22.1 轻度使用感受

前两天Google出了新的Support Library v22.1,这两天轻度使用了一下,贴点使用感受,慢慢更新

  1. Google又改Activity的基类了,原来是ActionbarActivity,现在是AppCompatActivity。很多逻辑也发生变化,之前是用Toolbar作为Actionbar的,现在也不需要了,直接自带Actionbar了,也是Material Design的设计,这个改动如果是从新写App的话挺好,如果是之前的老项目。。估计要改的吐血吧
  2. 各种Material Design的组件更新极其缓慢,这次确实又加了几个,但是设置不了深度,从用户的角度根本看不出来有多大变化。Annimation什么的通通没有,Google看这样子是不打算支持4.x用户了么?
  3. AlertDialog出了新的了,改下import就行了,好评。

发现一个有趣的东西:Docker

最近在写一个新的应用,后端用的是Nginx +  Python + Django + Gunicorn + Celery + libav,Celery又依赖RabbitMQ,为了让Celery和Gunicorn跑起来又用了Supervisor,东西太多配置又太繁琐,写代码调试和部署都是一个挑战,时间长了怕是配置文件和日志的location都要忘光光。我的工作环境是Mac和Windows,服务器则是Ubuntu或者Centos,像libav这种东西基本上算是Linux独有的,Mac虽然是*nix like但是毕竟不是Linux,想了半天最后决定放弃在Mac做Celery worker的调试,改在Ubuntu的服务器上用最原始的Log来分析调试,简直蛋疼到不行。

这两天在网上看到了Docker这个玩意,看起来确实不错:在Linux上有近乎Native的性能,在其它平台上则通过类似虚拟机的机制来构建运行环境,通过remote debug机制和IDE沟通,而部署则有点像拷贝虚拟机镜像:Docker的每一个优势都直击开发部署的痛点,下次部署的时候一定要用一下。

愿让我满怀期望的Docker,不要像Cygwin一样让我失望。

Overload vs Override, Java vs Groovy

Groovy is one of my favorite languages. It is simple yet powerful. More importantly, it can be integrated into Java stack easily, bringing us thousands of libraries that are ready to use.

One interesting thing about Java and Groovy is the difference in treating overload. A lot of scripting languages do not have function overloading. For example, in Python you need to use named parameters if you want to have some kinds of workaround for that. In JavaScript you need to check if a parameter is defined manually in the function, which brings complexity and lacks intuition. Same things happen to Matlab, where you have to check the number of arguments like the old C code:

Java has overloading, and it is reasonable that Groovy also has overloading. However, they treat overloading differently.

Java dispatch overloading during compile time. In other words, it is fixed and cannot be changed during run-time. However, as a dynamic language, Groovy dispatch overloading at run-time, which causes a different result in the following program.

These two pieces of code are identical to each other, except for the class name. However, the results are quite different. In Java, the output is String and Object, while in Groovy, the output is String and String.

What if we call the Java code in Groovy, and call the Groovy code in Java?

For the Java code, the output is String and Object, and for Groovy, String and String.

This experiment tells us the only deciding factor of which kind of dispatch method is being used is your code type that you are actually writing. If you are writing Java, not matter you are referring a code from Java or Groovy,  overloading functions are dispatched at compile time. If you are writing Groovy, overloading functions are dispatched at run time. This brings us a very essential problem: we need to be really careful with overloaded functions that come from Java library when we are writing Groovy code: it is the devil in the detail.

The good news is that, for function override, both Java and Groovy treat it in run-time and share the same behaviour.

改用CloudFlare的CDN并且启用了SSL

前段时间申请了StartSSL的免费SSL证书,但是由于Incapsula的免费版不支持SSL,也就一直没有部署。这两天看到Github的DDOS攻击事件,看到即使是一个毫不相关的网站,也有可能被别有用心的用来进行一些网络攻击,遂决定部署起来。

现在已经将CDN切换到了CloudFlare上,并且启用了Strict SSL,服务器端也设置了HTTPS的重定向,不过由于WordPress一直使用的是绝对地址来链接网站内容,所以还会有一些图片和视频的资源文件走HTTP,这个问题可能需要再过段时间想想怎么解决。

The DDOS attack against Github & HTTP Hijacking in China

It is well known that recently a large scale DDOS attack was carried out against Github. The obvious attacker: Great Fire Wall in China, did this by HTTP Hijacking.

A lot of Chinese websites use Baidu’s Analytic service to collect visiting information as an alternative to Google Analytics, which is not available in mainland China. The GFW hijacked the Baidu’s Analytic Javascript file of all in bounding requests, i.e. from outside China to inside China, and changed the content of it so that this piece of Javascript code will visit several certain Github pages every 2 seconds. This caused a heavy burden on Github’s servers and eventually became a DDOS attack in which GFW made use of millions of Chinese people who are outside China but visiting Chinese websites to achieve its goal.

HTTP hijacking is so common in China that it is not a surprise for me to hear that GFW eventually made use of this to attack Github. In China, every ISP is using HTTP hijacking to push ads to end users. In fact, some of the hijacking code is so ugly written that it will even hijack some API HTTP requests like Json response. It has been confirmed by some of the users of an open source android app which I contributed to. They reported this issue several days ago and such circumstances will only happen when using certain ISP service.

The reason why there are so many HTTP hijacking in China may lie in two reasons. First, the ISP market is not a free market and only some of nationally owned companies are allowed to provide Internet service. Second, few websites use HTTPS to deliver their contents, which made them vulnerable to MITM attacks.

 

搬瓦工已经没救了

一直以为搬瓦工最近速度越来越慢是因为港大的出口带宽吃不消了,今天有空在Digital Ocean上开了一个Singapore的VPS测试了一下,发现是搬瓦工的问题。

这个是新加坡到HostUs的:

Capture

这个是新加坡到搬瓦工的:

Capture2

这个是SFO到搬瓦工的:

Capture3

都是在西海岸,搬瓦工的跨洋线路慢成狗,西海岸内部连接倒是挺快但是对我没有半毛钱用处啊,再这样下去搬瓦工到期就可以扔掉了。