Like Share Discussion Bookmark Smile

J.J. Huang   2020-12-17   OpenRestry   瀏覽次數:

OpenResty - 第二章 | OpenResty 整合 lua zlib

前言

在開始之前建議先看完OpenResty - 第一章 | OpenResty 介绍/編譯安裝,下面的整合安裝會比較清楚。

因為專案的gatwayservice之間是透過gzip壓縮,而gatway所使用的是OpenResty,所以需要openresty lua zlib整合安裝,讓lua支援解壓服務端壓縮過的資料。

OpenResty在編譯安裝的時候就加入了lua支援,所以無需再對nginx進行改造。但lua下對gzip進行解壓,需要藉助一個庫:
lua- zliblua是一個和C語言結合緊密的指令碼語言,實際上lua-zlib就是一個C語言編寫的庫,我們現在需要做的就是將其編譯成一個動態連結庫zlib.so,讓lua來引用。

  • HTTP壓縮介紹

HTTP壓縮是一種內建到網頁伺服器和網頁客戶端中以改進傳輸速度和頻寬利用率的方式。
HTTP資料在從伺服器傳送前就已壓縮:相容的瀏覽器將在下載所需的格式前宣告支援何種方法給伺服器;不支援壓縮方法的瀏覽器將下載未經壓縮的資料。最常見的壓縮方案包括gzipDeflate,但可用方案的完整列表由IANA維護。此外,第三方可能開發新的方法並納入到其自身的產品,例如Google的面向HTTP共享字典壓縮(SDCH)方案就實現在Google Chrome瀏覽器和使用在Google的伺服器上。

Example Headers:

1
Accept-Encoding: gzip, deflate, br

安裝問題

透過網路上的教學openresty lua zlib整合安裝
在安裝的過程中發生了一些錯誤;

錯誤訊息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
$ make && make install
Scanning dependencies of target cmod_zlib
[ 50%] Building C object CMakeFiles/cmod_zlib.dir/lua_zlib.c.o
[100%] Linking C shared module zlib.so
Undefined symbols for architecture x86_64:
"_luaL_argerror", referenced from:
_lz_checksum in lua_zlib.c.o
"_luaL_buffinit", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_luaL_checklstring", referenced from:
_lz_checksum in lua_zlib.c.o
"_luaL_checkoption", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_luaL_newmetatable", referenced from:
_lz_create_deflate_mt in lua_zlib.c.o
_lz_create_inflate_mt in lua_zlib.c.o
"_luaL_optinteger", referenced from:
_lz_deflate_new in lua_zlib.c.o
"_luaL_prepbuffer", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_luaL_pushresult", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_luaL_register", referenced from:
_luaopen_zlib in lua_zlib.c.o
"_lua_call", referenced from:
_lz_checksum in lua_zlib.c.o
"_lua_concat", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_lua_error", referenced from:
_lz_assert in lua_zlib.c.o
_lz_filter_impl in lua_zlib.c.o
"_lua_getfield", referenced from:
_lz_deflate_new in lua_zlib.c.o
_lz_inflate_new in lua_zlib.c.o
"_lua_gettop", referenced from:
_lz_filter_impl in lua_zlib.c.o
_lz_checksum in lua_zlib.c.o
"_lua_isnumber", referenced from:
_lz_inflate_new in lua_zlib.c.o
_lz_checksum in lua_zlib.c.o
"_lua_isstring", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_lua_newuserdata", referenced from:
_lz_deflate_new in lua_zlib.c.o
_lz_inflate_new in lua_zlib.c.o
_lz_version in lua_zlib.c.o
"_lua_pushboolean", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_lua_pushcclosure", referenced from:
_lz_create_deflate_mt in lua_zlib.c.o
_lz_create_inflate_mt in lua_zlib.c.o
_lz_deflate_new in lua_zlib.c.o
_lz_inflate_new in lua_zlib.c.o
_lz_checksum_new in lua_zlib.c.o
"_lua_pushfstring", referenced from:
_lz_assert in lua_zlib.c.o
_lz_filter_impl in lua_zlib.c.o
"_lua_pushinteger", referenced from:
_luaopen_zlib in lua_zlib.c.o
_lz_filter_impl in lua_zlib.c.o
"_lua_pushlightuserdata", referenced from:
_lz_checksum_new in lua_zlib.c.o
"_lua_pushlstring", referenced from:
_luaopen_zlib in lua_zlib.c.o
_lz_filter_impl in lua_zlib.c.o
"_lua_pushnil", referenced from:
_lz_deflate_new in lua_zlib.c.o
_lz_inflate_new in lua_zlib.c.o
_lz_filter_impl in lua_zlib.c.o
"_lua_pushnumber", referenced from:
_lz_version in lua_zlib.c.o
_lz_checksum_new in lua_zlib.c.o
_lz_checksum in lua_zlib.c.o
"_lua_pushstring", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_lua_pushvalue", referenced from:
_lz_filter_impl in lua_zlib.c.o
_lz_checksum in lua_zlib.c.o
"_lua_replace", referenced from:
_lz_filter_impl in lua_zlib.c.o
_lz_checksum in lua_zlib.c.o
"_lua_setfield", referenced from:
_lz_create_deflate_mt in lua_zlib.c.o
_lz_create_inflate_mt in lua_zlib.c.o
"_lua_setmetatable", referenced from:
_lz_deflate_new in lua_zlib.c.o
_lz_inflate_new in lua_zlib.c.o
_lz_filter_impl in lua_zlib.c.o
"_lua_settable", referenced from:
_luaopen_zlib in lua_zlib.c.o
"_lua_settop", referenced from:
_lz_create_deflate_mt in lua_zlib.c.o
_lz_create_inflate_mt in lua_zlib.c.o
"_lua_tointeger", referenced from:
_lz_inflate_new in lua_zlib.c.o
"_lua_tolstring", referenced from:
_lz_filter_impl in lua_zlib.c.o
"_lua_tonumber", referenced from:
_lz_checksum in lua_zlib.c.o
"_lua_touserdata", referenced from:
_lz_deflate_delete in lua_zlib.c.o
_lz_inflate_delete in lua_zlib.c.o
_lz_filter_impl in lua_zlib.c.o
_lz_checksum in lua_zlib.c.o
"_lua_type", referenced from:
_lz_filter_impl in lua_zlib.c.o
_lz_checksum in lua_zlib.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [zlib.so] Error 1
make[1]: *** [CMakeFiles/cmod_zlib.dir/all] Error 2
make: *** [all] Error 2

解決

  • Mac OS X (macOS) 用户
    1. 準備&檢查,確保 gcc, git, make, cmake皆已安裝
    • 檢查gcc是否有安裝
      1
      2
      3
      4
      5
      6
      $ gcc -v
      Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
      Apple clang version 11.0.3 (clang-1103.0.32.62)
      Target: x86_64-apple-darwin19.5.0
      Thread model: posix
      InstalledDir: /Library/Developer/CommandLineTools/usr/bin
    • 檢查git是否有安裝
      1
      2
      $ git --version
      git version 2.24.3 (Apple Git-128)
    • 檢查make是否有安裝
      1
      2
      3
      4
      5
      6
      7
      8
      $ make -v
      GNU Make 3.81
      Copyright (C) 2006 Free Software Foundation, Inc.
      This is free software; see the source for copying conditions.
      There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
      PARTICULAR PURPOSE.

      This program built for i386-apple-darwin11.3.0
    • 檢查cmake是否有安裝
      1
      2
      3
      4
      $ cmake -version
      cmake version 3.19.1

      CMake suite maintained and supported by Kitware (kitware.com/cmake).

      註:如有缺少,可使用Homebrew做安裝。

    1. 下載zlib原始碼
      1
      2
      Download page:https://zlib.net/
      Downlaod link:https://zlib.net/zlib-1.2.11.tar.gz

      註:文章撰寫時最新版為:zlib-1.2.11

    1. 解壓縮[zlib-1.2.11.tar.gz]
      1
      $ tar -xzvf zlib-1.2.11.tar.gz

      註:zlib-1.2.11.tar.gz解壓縮路徑自己選擇即可,不影響過程,這邊解壓縮至/Users/morose/Documents/Temp/tutorial/zlib-1.2.11

    1. 進入到步驟3zlib-1.2.11解壓縮後的目錄下。
      1
      $ cd /Users/morose/Documents/Temp/tutorial/zlib-1.2.11
      當前zlib-1.2.11目錄列表:
    1. ./configure (配置)
      1
      $ ./configure
      output:
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      Checking for gcc...
      Checking for shared library support...
      Building shared library libz.1.2.11.dylib with gcc.
      Checking for size_t... Yes.
      Checking for off64_t... No.
      Checking for fseeko... Yes.
      Checking for strerror... Yes.
      Checking for unistd.h... Yes.
      Checking for stdarg.h... Yes.
      Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().
      Checking for vsnprintf() in stdio.h... Yes.
      Checking for return value of vsnprintf()... Yes.
      Checking for attribute(visibility) support... Yes.
      當前zlib-1.2.11目錄列表:
    1. make (製作)
      1
      $ make
      output:
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      gcc -O3 -DHAVE_HIDDEN -I. -c -o example.o test/example.c
      gcc -O3 -DHAVE_HIDDEN -c -o adler32.o adler32.c
      gcc -O3 -DHAVE_HIDDEN -c -o crc32.o crc32.c
      gcc -O3 -DHAVE_HIDDEN -c -o deflate.o deflate.c
      gcc -O3 -DHAVE_HIDDEN -c -o infback.o infback.c
      gcc -O3 -DHAVE_HIDDEN -c -o inffast.o inffast.c
      gcc -O3 -DHAVE_HIDDEN -c -o inflate.o inflate.c
      gcc -O3 -DHAVE_HIDDEN -c -o inftrees.o inftrees.c
      gcc -O3 -DHAVE_HIDDEN -c -o trees.o trees.c
      gcc -O3 -DHAVE_HIDDEN -c -o zutil.o zutil.c
      gcc -O3 -DHAVE_HIDDEN -c -o compress.o compress.c
      gcc -O3 -DHAVE_HIDDEN -c -o uncompr.o uncompr.c
      gcc -O3 -DHAVE_HIDDEN -c -o gzclose.o gzclose.c
      gcc -O3 -DHAVE_HIDDEN -c -o gzlib.o gzlib.c
      gcc -O3 -DHAVE_HIDDEN -c -o gzread.o gzread.c
      gcc -O3 -DHAVE_HIDDEN -c -o gzwrite.o gzwrite.c
      libtool -o libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o
      gcc -O3 -DHAVE_HIDDEN -o example example.o -L. libz.a
      gcc -O3 -DHAVE_HIDDEN -I. -c -o minigzip.o test/minigzip.c
      gcc -O3 -DHAVE_HIDDEN -o minigzip minigzip.o -L. libz.a
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/adler32.o adler32.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/crc32.o crc32.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/deflate.o deflate.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/infback.o infback.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/inffast.o inffast.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/inflate.o inflate.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/inftrees.o inftrees.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/trees.o trees.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/zutil.o zutil.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/compress.o compress.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/uncompr.o uncompr.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/gzclose.o gzclose.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/gzlib.o gzlib.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/gzread.o gzread.c
      gcc -O3 -fPIC -DHAVE_HIDDEN -DPIC -c -o objs/gzwrite.o gzwrite.c
      gcc -dynamiclib -install_name /usr/local/lib/libz.1.dylib -compatibility_version 1 -current_version 1.2.11 -O3 -fPIC -DHAVE_HIDDEN -o libz.1.2.11.dylib adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo -lc
      rm -f libz.dylib libz.1.dylib
      ln -s libz.1.2.11.dylib libz.dylib
      ln -s libz.1.2.11.dylib libz.1.dylib
      gcc -O3 -DHAVE_HIDDEN -o examplesh example.o -L. libz.1.2.11.dylib
      gcc -O3 -DHAVE_HIDDEN -o minigzipsh minigzip.o -L. libz.1.2.11.dylib
      當前zlib-1.2.11目錄列表:

      註:執行後應會產生出libz.a

    1. 下載zlib原始碼
      1
      2
      Download page:https://github.com/brimworks/lua-zlib
      Downlaod link:https://github.com/brimworks/lua-zlib/archive/master.zip
    1. 解壓縮[lua-zlib-master.zip]
      1
      $ unzip lua-zlib-master.zip

      註:lua-zlib-master.zip解壓縮路徑自己選擇即可,不影響過程,這邊解壓縮至/Users/morose/Documents/Temp/tutorial/lua-zlib-master

    1. 將第6步驟產生的libz.a複製至lua-zlib-master目錄下。
      1
      $ cp /Users/morose/Documents/Temp/tutorial/zlib-1.2.11/libz.a /Users/morose/Documents/Temp/tutorial/lua-zlib-master
    1. 進入到步驟8lua-zlib-master解壓縮後的目錄下。
      1
      $ cd /Users/morose/Documents/Temp/tutorial/lua-zlib-master
    1. 執行gcc編譯lua_zlib.o
      1
      $ gcc -O2 -c -o lua_zlib.o lua_zlib.c -I${luajit-2.1} -I${zlib-1.2.11}

      註:${luajit-2.1} = 當初所編譯後的openresty的路徑下的/luajit/include/luajit-2.1檔路徑;${zlib-1.2.11} = zlib-1.2.11解壓縮的路徑。

      1
      2
      # Command sample
      $ gcc -O2 -c -o lua_zlib.o lua_zlib.c -I/Users/morose/Documents/Temp/tutorial/openresty/luajit/include/luajit-2.1 -I/Users/morose/Documents/Temp/tutorial/zlib-1.2.11

      註:執行後應會產生出lua_zlib.o

    1. 執行gcc編譯zlib.so
      1
      $ gcc -shared -o zlib.so lua_zlib.o libz.a ${libluajit-5.1.a}

      註:${libluajit-5.1.a} = 當初所編譯後的openresty的路徑下的/luajit/lib/libluajit-5.1.a檔路徑

      1
      2
      # Command sample
      $ gcc -shared -o zlib.so lua_zlib.o libz.a /Users/morose/Documents/Temp/tutorial/openresty/luajit/lib/libluajit-5.1.a

      註:執行後應會產生出zlib.so

    1. zlib.so複製進當初所編譯後的openresty/lualib/下。
      1
      $ cp /Users/morose/Documents/Temp/tutorial/lua-zlib-master/zlib.so /Users/morose/Documents/Temp/tutorial/openresty/lualib
    1. 恭喜已經整合完畢。

  • Windwos 用户

  • 因文獻甚少這邊補上Windwos的相關文獻。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
windows上安裝lua-zlib
準備
1 安裝 MinGW,配置好path 2 下載zlib-1.2.11源碼 3 下載lua-zlib源碼

進入 zlib-1.2.11 目錄

mingw32-make -f win32/Makefile.gcc
正常或生成幾個文件,libz.a這個文件備用

進入 lua-zlib目錄

mingw32-gcc -O2 -c -o lua_zlib.o lua_zlib.c -I(這裡接luajit\include\luajit-2.1) -I(這裡接前面的zlib目錄)
拷貝之前生成的libz文件到當前目前
mingw32-gcc -shared -o zlib.dll lua_zlib.o libz.a (這裡接lua51.dll的地址)
正常應該會生成zlib.dll文件這個就是最終要的文件

註:文章轉於kooky126的博客, windows上安装lua-zlib

補充

如果沒有整合zlib在使用的時候Server端有正常回應給gatway,但是無法解壓回來,會顯示status 500 Internal Server Error
這補一張Postman示範:

  • Status 500
  • Status 200

結語

這盡量依照手把手教學的想法寫完這篇文章,希望在使用OpenResty整合lua zlib上遇到有這方面需求的人可以得到幫助。原創文章撰寫不易,還請多多支持,謝謝。


註:以上參考了
Wiki HTTP壓縮
openresty lua zlib整合安裝 讓lua支援解壓服務端壓縮過的資料