Nginx Forum - Nginx Development Nginx development mailing list [read only] http://www.ldmicj.icu/list.php?29 Mon, 30 Nov 2020 05:06:39 -0500 Phorum 5.2.16 http://www.ldmicj.icu/read.php?29,290090,290090#msg-290090 [PATCH] Replaced loop with __builtin_ctzl for detecting first 0 in bitnamp (1 reply) http://www.ldmicj.icu/read.php?29,290090,290090#msg-290090 # User Noah Goldstein <goldstein.w.n@gmail.com>
# Date 1606497081 18000
# Fri Nov 27 12:11:21 2020 -0500
# Node ID 7ec2fc7b29d6614df28152dd4a895e6139138890
# Parent 90cc7194e993f8d722347e9f46a00f65dffc3935
Replaced loop with __builtin_ctzl for detecting first 0 in bitnamp
No particular pressing reason for this change other than the performance benefit it yields. Converts a loop that could take 63 iterations (and had a branch) to 5 instructions

diff -r 90cc7194e993 -r 7ec2fc7b29d6 src/core/ngx_slab.c
--- a/src/core/ngx_slab.c Fri Nov 27 00:01:20 2020 +0300
+++ b/src/core/ngx_slab.c Fri Nov 27 12:11:21 2020 -0500
@@ -235,36 +235,33 @@

if (bitmap[n] != NGX_SLAB_BUSY) {

- for (m = 1, i = 0; m; m <<= 1, i++) {
- if (bitmap[n] & m) {
- continue;
+ m = ~bitmap[n];
+ i = __builtin_ctzl(m);
+
+ bitmap[n] = ~(m & (m - 1));
+
+ i = (n * 8 * sizeof(uintptr_t) + i) << shift;
+
+ p = (uintptr_t) bitmap + i;
+
+ pool->stats[slot].used++;
+
+ if (bitmap[n] == NGX_SLAB_BUSY) {
+ for (n = n + 1; n < map; n++) {
+ if (bitmap[n] != NGX_SLAB_BUSY) {
+ goto done;
+ }
}

- bitmap[n] |= m;
-
- i = (n * 8 * sizeof(uintptr_t) + i) << shift;
-
- p = (uintptr_t) bitmap + i;
-
- pool->stats[slot].used++;
+ prev = ngx_slab_page_prev(page);
+ prev->next = page->next;
+ page->next->prev = page->prev;

- if (bitmap[n] == NGX_SLAB_BUSY) {
- for (n = n + 1; n < map; n++) {
- if (bitmap[n] != NGX_SLAB_BUSY) {
- goto done;
- }
- }
+ page->next = NULL;
+ page->prev = NGX_SLAB_SMALL;
+ }

- prev = ngx_slab_page_prev(page);
- prev->next = page->next;
- page->next->prev = page->prev;
-
- page->next = NULL;
- page->prev = NGX_SLAB_SMALL;
- }
-
- goto done;
- }
+ goto done;
}
}

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Anonymous User Nginx Development Fri, 27 Nov 2020 15:18:12 -0500
http://www.ldmicj.icu/read.php?29,290087,290087#msg-290087 [njs] HTTP: renaming reqBody, resBody to requestBuffer and responseBuffer. (no replies) http://www.ldmicj.icu/read.php?29,290087,290087#msg-290087 branches:
changeset: 1576:a8cb5f47bfea
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Fri Nov 27 12:28:44 2020 +0000
description:
HTTP: renaming reqBody,resBody to requestBuffer and responseBuffer.

In 434f20c29f4c, r.reqBody and r.resBody were introduced. Since quite
often request body and response body are valid strings, it is preferable
to leave both variants to avoid potential conversions.

To make distinction clearer, requestText and responseText aliases to
requestBody and responseBody were also added.

diffstat:

nginx/ngx_http_js_module.c | 22 ++++++++++++++++++++--
1 files changed, 20 insertions(+), 2 deletions(-)

diffs (45 lines):

diff -r fac7e5dc8009 -r a8cb5f47bfea nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c Thu Nov 26 21:43:17 2020 +0300
+++ b/nginx/ngx_http_js_module.c Fri Nov 27 12:28:44 2020 +0000
@@ -328,10 +328,19 @@ static njs_external_t ngx_http_js_ext_r

{
.flags = NJS_EXTERN_PROPERTY,
- .name.string = njs_str("reqBody"),
+ .name.string = njs_str("requestText"),
.enumerable = 1,
.u.property = {
.handler = ngx_http_js_ext_get_request_body,
+ .magic32 = NGX_JS_STRING,
+ }
+ },
+
+ {
+ .flags = NJS_EXTERN_PROPERTY,
+ .name.string = njs_str("requestBuffer"),
+ .u.property = {
+ .handler = ngx_http_js_ext_get_request_body,
.magic32 = NGX_JS_BUFFER,
}
},
@@ -355,10 +364,19 @@ static njs_external_t ngx_http_js_ext_r

{
.flags = NJS_EXTERN_PROPERTY,
- .name.string = njs_str("resBody"),
+ .name.string = njs_str("responseText"),
.enumerable = 1,
.u.property = {
.handler = ngx_http_js_ext_get_response_body,
+ .magic32 = NGX_JS_STRING,
+ }
+ },
+
+ {
+ .flags = NJS_EXTERN_PROPERTY,
+ .name.string = njs_str("responseBuffer"),
+ .u.property = {
+ .handler = ngx_http_js_ext_get_response_body,
.magic32 = NGX_JS_BUFFER,
}
},
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Fri, 27 Nov 2020 08:14:09 -0500
http://www.ldmicj.icu/read.php?29,290083,290083#msg-290083 [nginx] Upstream: excluded down servers from the next_upstream tries. (no replies) http://www.ldmicj.icu/read.php?29,290083,290083#msg-290083 branches:
changeset: 7750:90cc7194e993
user: Ruslan Ermilov <ru@nginx.com>
date: Fri Nov 27 00:01:20 2020 +0300
description:
Upstream: excluded down servers from the next_upstream tries.

Previously, the number of next_upstream tries included servers marked
as "down", resulting in "no live upstreams" with the code 502 instead
of the code derived from an attempt to connect to the last tried "up"
server (ticket #2096).

diffstat:

src/http/ngx_http_upstream_round_robin.c | 20 +++++++++++++++++---
src/http/ngx_http_upstream_round_robin.h | 1 +
src/stream/ngx_stream_upstream_round_robin.c | 20 +++++++++++++++++---
src/stream/ngx_stream_upstream_round_robin.h | 1 +
4 files changed, 36 insertions(+), 6 deletions(-)

diffs (208 lines):

diff -r ac09a57ec50d -r 90cc7194e993 src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c Thu Nov 26 23:46:59 2020 +0300
+++ b/src/http/ngx_http_upstream_round_robin.c Fri Nov 27 00:01:20 2020 +0300
@@ -10,8 +10,8 @@
#include <ngx_http.h>


-#define ngx_http_upstream_tries(p) ((p)->number \
- + ((p)->next ? (p)->next->number : 0))
+#define ngx_http_upstream_tries(p) ((p)->tries \
+ + ((p)->next ? (p)->next->tries : 0))


static ngx_http_upstream_rr_peer_t *ngx_http_upstream_get_peer(
@@ -32,7 +32,7 @@ ngx_http_upstream_init_round_robin(ngx_c
ngx_http_upstream_srv_conf_t *us)
{
ngx_url_t u;
- ngx_uint_t i, j, n, w;
+ ngx_uint_t i, j, n, w, t;
ngx_http_upstream_server_t *server;
ngx_http_upstream_rr_peer_t *peer, **peerp;
ngx_http_upstream_rr_peers_t *peers, *backup;
@@ -44,6 +44,7 @@ ngx_http_upstream_init_round_robin(ngx_c

n = 0;
w = 0;
+ t = 0;

for (i = 0; i < us->servers->nelts; i++) {
if (server[i].backup) {
@@ -52,6 +53,10 @@ ngx_http_upstream_init_round_robin(ngx_c

n += server[i].naddrs;
w += server[i].naddrs * server[i].weight;
+
+ if (!server[i].down) {
+ t += server[i].naddrs;
+ }
}

if (n == 0) {
@@ -75,6 +80,7 @@ ngx_http_upstream_init_round_robin(ngx_c
peers->number = n;
peers->weighted = (w != n);
peers->total_weight = w;
+ peers->tries = t;
peers->name = &us->host;

n = 0;
@@ -110,6 +116,7 @@ ngx_http_upstream_init_round_robin(ngx_c

n = 0;
w = 0;
+ t = 0;

for (i = 0; i < us->servers->nelts; i++) {
if (!server[i].backup) {
@@ -118,6 +125,10 @@ ngx_http_upstream_init_round_robin(ngx_c

n += server[i].naddrs;
w += server[i].naddrs * server[i].weight;
+
+ if (!server[i].down) {
+ t += server[i].naddrs;
+ }
}

if (n == 0) {
@@ -139,6 +150,7 @@ ngx_http_upstream_init_round_robin(ngx_c
backup->number = n;
backup->weighted = (w != n);
backup->total_weight = w;
+ backup->tries = t;
backup->name = &us->host;

n = 0;
@@ -214,6 +226,7 @@ ngx_http_upstream_init_round_robin(ngx_c
peers->number = n;
peers->weighted = 0;
peers->total_weight = n;
+ peers->tries = n;
peers->name = &us->host;

peerp = &peers->peer;
@@ -332,6 +345,7 @@ ngx_http_upstream_create_round_robin_pee

peers->single = (ur->naddrs == 1);
peers->number = ur->naddrs;
+ peers->tries = ur->naddrs;
peers->name = &ur->host;

if (ur->sockaddr) {
diff -r ac09a57ec50d -r 90cc7194e993 src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h Thu Nov 26 23:46:59 2020 +0300
+++ b/src/http/ngx_http_upstream_round_robin.h Fri Nov 27 00:01:20 2020 +0300
@@ -68,6 +68,7 @@ struct ngx_http_upstream_rr_peers_s {
#endif

ngx_uint_t total_weight;
+ ngx_uint_t tries;

unsigned single:1;
unsigned weighted:1;
diff -r ac09a57ec50d -r 90cc7194e993 src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c Thu Nov 26 23:46:59 2020 +0300
+++ b/src/stream/ngx_stream_upstream_round_robin.c Fri Nov 27 00:01:20 2020 +0300
@@ -10,8 +10,8 @@
#include <ngx_stream.h>


-#define ngx_stream_upstream_tries(p) ((p)->number \
- + ((p)->next ? (p)->next->number : 0))
+#define ngx_stream_upstream_tries(p) ((p)->tries \
+ + ((p)->next ? (p)->next->tries : 0))


static ngx_stream_upstream_rr_peer_t *ngx_stream_upstream_get_peer(
@@ -38,7 +38,7 @@ ngx_stream_upstream_init_round_robin(ngx
ngx_stream_upstream_srv_conf_t *us)
{
ngx_url_t u;
- ngx_uint_t i, j, n, w;
+ ngx_uint_t i, j, n, w, t;
ngx_stream_upstream_server_t *server;
ngx_stream_upstream_rr_peer_t *peer, **peerp;
ngx_stream_upstream_rr_peers_t *peers, *backup;
@@ -50,6 +50,7 @@ ngx_stream_upstream_init_round_robin(ngx

n = 0;
w = 0;
+ t = 0;

for (i = 0; i < us->servers->nelts; i++) {
if (server[i].backup) {
@@ -58,6 +59,10 @@ ngx_stream_upstream_init_round_robin(ngx

n += server[i].naddrs;
w += server[i].naddrs * server[i].weight;
+
+ if (!server[i].down) {
+ t += server[i].naddrs;
+ }
}

if (n == 0) {
@@ -81,6 +86,7 @@ ngx_stream_upstream_init_round_robin(ngx
peers->number = n;
peers->weighted = (w != n);
peers->total_weight = w;
+ peers->tries = t;
peers->name = &us->host;

n = 0;
@@ -116,6 +122,7 @@ ngx_stream_upstream_init_round_robin(ngx

n = 0;
w = 0;
+ t = 0;

for (i = 0; i < us->servers->nelts; i++) {
if (!server[i].backup) {
@@ -124,6 +131,10 @@ ngx_stream_upstream_init_round_robin(ngx

n += server[i].naddrs;
w += server[i].naddrs * server[i].weight;
+
+ if (!server[i].down) {
+ t += server[i].naddrs;
+ }
}

if (n == 0) {
@@ -145,6 +156,7 @@ ngx_stream_upstream_init_round_robin(ngx
backup->number = n;
backup->weighted = (w != n);
backup->total_weight = w;
+ backup->tries = t;
backup->name = &us->host;

n = 0;
@@ -220,6 +232,7 @@ ngx_stream_upstream_init_round_robin(ngx
peers->number = n;
peers->weighted = 0;
peers->total_weight = n;
+ peers->tries = n;
peers->name = &us->host;

peerp = &peers->peer;
@@ -342,6 +355,7 @@ ngx_stream_upstream_create_round_robin_p

peers->single = (ur->naddrs == 1);
peers->number = ur->naddrs;
+ peers->tries = ur->naddrs;
peers->name = &ur->host;

if (ur->sockaddr) {
diff -r ac09a57ec50d -r 90cc7194e993 src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h Thu Nov 26 23:46:59 2020 +0300
+++ b/src/stream/ngx_stream_upstream_round_robin.h Fri Nov 27 00:01:20 2020 +0300
@@ -66,6 +66,7 @@ struct ngx_stream_upstream_rr_peers_s {
#endif

ngx_uint_t total_weight;
+ ngx_uint_t tries;

unsigned single:1;
unsigned weighted:1;
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
ru@nginx.com Nginx Development Thu, 26 Nov 2020 16:10:14 -0500
http://www.ldmicj.icu/read.php?29,290082,290082#msg-290082 [nginx] Version bump. (no replies) http://www.ldmicj.icu/read.php?29,290082,290082#msg-290082 branches:
changeset: 7749:ac09a57ec50d
user: Ruslan Ermilov <ru@nginx.com>
date: Thu Nov 26 23:46:59 2020 +0300
description:
Version bump.

diffstat:

src/core/nginx.h | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 66a441bf669b -r ac09a57ec50d src/core/nginx.h
--- a/src/core/nginx.h Tue Nov 24 18:06:34 2020 +0300
+++ b/src/core/nginx.h Thu Nov 26 23:46:59 2020 +0300
@@ -9,8 +9,8 @@
#define _NGINX_H_INCLUDED_


-#define nginx_version 1019005
-#define NGINX_VERSION "1.19.5"
+#define nginx_version 1019006
+#define NGINX_VERSION "1.19.6"
#define NGINX_VER "nginx/" NGINX_VERSION

#ifdef NGX_BUILD
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
ru@nginx.com Nginx Development Thu, 26 Nov 2020 16:10:14 -0500
http://www.ldmicj.icu/read.php?29,290081,290081#msg-290081 [PATCH] Core: fixed inconsistent state of subrequest's headers_in list (3 replies) http://www.ldmicj.icu/read.php?29,290081,290081#msg-290081 # User Jan Prachař <jan.prachar@gmail.com>
# Date 1606420825 -3600
# Thu Nov 26 21:00:25 2020 +0100
# Node ID cf3d537ec6706f8713a757df256f2cfccb8f9b01
# Parent e35b529b03781e64912e0d8a72bd0f957dc08cd2
Core: fixed inconsistent state of subrequest's headers_in list

When copying structure ngx_list_t, a reference to the last part need to be
updated, if list contains only one part.

This fixes an issue, when adding a header to the subrequest's headers_in list
has no effect.

diff -r e35b529b0378 -r cf3d537ec670 src/core/ngx_list.h
--- a/src/core/ngx_list.h Mon Sep 21 19:49:49 2020 +0200
+++ b/src/core/ngx_list.h Thu Nov 26 21:00:25 2020 +0100
@@ -51,6 +51,15 @@
return NGX_OK;
}

+static ngx_inline void
+ngx_list_copy(ngx_list_t *target, ngx_list_t *src)
+{
+ *target = *src;
+ if (target->part.next == NULL) {
+ target->last = &target->part;
+ }
+}
+

/*
*
diff -r e35b529b0378 -r cf3d537ec670 src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c Mon Sep 21 19:49:49 2020 +0200
+++ b/src/http/ngx_http_core_module.c Thu Nov 26 21:00:25 2020 +0100
@@ -2364,6 +2364,7 @@
sr->pool = r->pool;

sr->headers_in = r->headers_in;
+ ngx_list_copy(&sr->headers_in.headers, &r->headers_in.headers);

ngx_http_clear_content_length(sr);
ngx_http_clear_accept_ranges(sr);

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Jan Prachař Nginx Development Fri, 27 Nov 2020 15:40:05 -0500
http://www.ldmicj.icu/read.php?29,290080,290080#msg-290080 [njs] Modules: fixed promise events handling. (no replies) http://www.ldmicj.icu/read.php?29,290080,290080#msg-290080 branches:
changeset: 1575:fac7e5dc8009
user: Alexander Borisov <alexander.borisov@nginx.com>
date: Thu Nov 26 21:43:17 2020 +0300
description:
Modules: fixed promise events handling.

Previously, promise chain might not be invoked at all in some cases.
Specifically, this happened in HTTP module if promise chain did not start
with a r.subrequest() invocation.

The fix is to always process all pending promise events after the main module
function.

This closes #359 issue on GitHub.

diffstat:

nginx/ngx_js.c | 6 +-----
src/njs_vm.c | 2 +-
2 files changed, 2 insertions(+), 6 deletions(-)

diffs (28 lines):

diff -r a141a29417dc -r fac7e5dc8009 nginx/ngx_js.c
--- a/nginx/ngx_js.c Thu Nov 26 11:36:03 2020 +0000
+++ b/nginx/ngx_js.c Thu Nov 26 21:43:17 2020 +0300
@@ -79,11 +79,7 @@ ngx_js_call(njs_vm_t *vm, ngx_str_t *fna
return NGX_ERROR;
}

- if (njs_vm_pending(vm)) {
- return NGX_AGAIN;
- }
-
- return NGX_OK;
+ return njs_vm_run(vm);
}


diff -r a141a29417dc -r fac7e5dc8009 src/njs_vm.c
--- a/src/njs_vm.c Thu Nov 26 11:36:03 2020 +0000
+++ b/src/njs_vm.c Thu Nov 26 21:43:17 2020 +0300
@@ -578,7 +578,7 @@ njs_vm_handle_events(njs_vm_t *vm)

} while (!njs_queue_is_empty(promise_events));

- return njs_posted_events(vm) ? NJS_AGAIN : NJS_OK;
+ return njs_vm_pending(vm) ? NJS_AGAIN : NJS_OK;
}


_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Alexander Borisov Nginx Development Thu, 26 Nov 2020 13:46:10 -0500
http://www.ldmicj.icu/read.php?29,290077,290077#msg-290077 [njs] Modules: renaming vars to rawVariables to better reflect purpose. (no replies) http://www.ldmicj.icu/read.php?29,290077,290077#msg-290077 branches:
changeset: 1574:a141a29417dc
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Thu Nov 26 11:36:03 2020 +0000
description:
Modules: renaming vars to rawVariables to better reflect purpose.

In 434f20c29f4c, obj.vars was introduced. obj.vars is almost identical
to obj.variables except a value of the Buffer type is returned.

Since most nginx variables are valid strings, it is preferable to
leave both variants.

To avoid confusion rawVariables name is used for Buffer variables.

diffstat:

nginx/ngx_http_js_module.c | 2 +-
nginx/ngx_stream_js_module.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

diffs (24 lines):

diff -r 42dfbf020c68 -r a141a29417dc nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c Thu Nov 26 11:11:01 2020 +0000
+++ b/nginx/ngx_http_js_module.c Thu Nov 26 11:36:03 2020 +0000
@@ -406,7 +406,7 @@ static njs_external_t ngx_http_js_ext_r

{
.flags = NJS_EXTERN_OBJECT,
- .name.string = njs_str("vars"),
+ .name.string = njs_str("rawVariables"),
.u.object = {
.writable = 1,
.prop_handler = ngx_http_js_ext_variables,
diff -r 42dfbf020c68 -r a141a29417dc nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c Thu Nov 26 11:11:01 2020 +0000
+++ b/nginx/ngx_stream_js_module.c Thu Nov 26 11:36:03 2020 +0000
@@ -245,7 +245,7 @@ static njs_external_t ngx_stream_js_ext

{
.flags = NJS_EXTERN_OBJECT,
- .name.string = njs_str("vars"),
+ .name.string = njs_str("rawVariables"),
.u.object = {
.writable = 1,
.prop_handler = ngx_stream_js_ext_variables,
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Thu, 26 Nov 2020 06:40:05 -0500
http://www.ldmicj.icu/read.php?29,290076,290076#msg-290076 [njs] Types: added description for "ngx" object. (no replies) http://www.ldmicj.icu/read.php?29,290076,290076#msg-290076 branches:
changeset: 1573:42dfbf020c68
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Thu Nov 26 11:11:01 2020 +0000
description:
Types: added description for "ngx" object.

diffstat:

test/ts/test.ts | 9 +++++++--
ts/ngx_core.d.ts | 14 ++++++++++++++
ts/ngx_http_js_module.d.ts | 1 +
ts/ngx_stream_js_module.d.ts | 1 +
4 files changed, 23 insertions(+), 2 deletions(-)

diffs (62 lines):

diff -r 5bd78c74777a -r 42dfbf020c68 test/ts/test.ts
--- a/test/ts/test.ts Thu Nov 26 11:10:59 2020 +0000
+++ b/test/ts/test.ts Thu Nov 26 11:11:01 2020 +0000
@@ -66,7 +66,6 @@ function http_module(r: NginxHTTPRequest
// Warning: vod = r.subrequest('/p/sub9', {detached:true}, reply => r.return(reply.status));
r.subrequest('/p/sub6', 'a=1&b=2').then(reply => r.return(reply.status,
JSON.stringify(JSON.parse(reply.responseBody ?? ''))));
-
}

function fs_module() {
@@ -107,7 +106,13 @@ function buffer(b: Buffer) {
b.equals(b);
}

-function builtins() {
+function njs_object() {
njs.dump('asdf');
njs.version != process.argv[1];
}
+
+function ngx_object() {
+ ngx.log(ngx.INFO, 'asdf');
+ ngx.log(ngx.WARN, Buffer.from('asdf'));
+ ngx.log(ngx.ERR, 'asdf');
+}
diff -r 5bd78c74777a -r 42dfbf020c68 ts/ngx_core.d.ts
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/ts/ngx_core.d.ts Thu Nov 26 11:11:01 2020 +0000
@@ -0,0 +1,14 @@
+interface NgxObject {
+ readonly INFO: number;
+ readonly WARN: number;
+ readonly ERR: number;
+ /**
+ * Writes a string to the error log with the specified level
+ * of logging.
+ * @param level Log level (ngx.INFO, ngx.WARN, ngx.ERR).
+ * @param message Message to log.
+ */
+ log(level: number, message: NjsStringOrBuffer): void;
+}
+
+declare const ngx: NgxObject;
diff -r 5bd78c74777a -r 42dfbf020c68 ts/ngx_http_js_module.d.ts
--- a/ts/ngx_http_js_module.d.ts Thu Nov 26 11:10:59 2020 +0000
+++ b/ts/ngx_http_js_module.d.ts Thu Nov 26 11:11:01 2020 +0000
@@ -1,4 +1,5 @@
/// <reference path="index.d.ts" />
+/// <reference path="ngx_core.d.ts" />

interface NginxHTTPArgs {
readonly [prop: string]: NjsByteString;
diff -r 5bd78c74777a -r 42dfbf020c68 ts/ngx_stream_js_module.d.ts
--- a/ts/ngx_stream_js_module.d.ts Thu Nov 26 11:10:59 2020 +0000
+++ b/ts/ngx_stream_js_module.d.ts Thu Nov 26 11:11:01 2020 +0000
@@ -1,4 +1,5 @@
/// <reference path="index.d.ts" />
+/// <reference path="ngx_core.d.ts" />

interface NginxStreamVariables {
readonly 'binary_remote_addr'?: NjsByteString;
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Thu, 26 Nov 2020 06:14:13 -0500
http://www.ldmicj.icu/read.php?29,290075,290075#msg-290075 [njs] Types: extending data types for methods with NjsStringLike args. (no replies) http://www.ldmicj.icu/read.php?29,290075,290075#msg-290075 branches:
changeset: 1572:5bd78c74777a
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Thu Nov 26 11:10:59 2020 +0000
description:
Types: extending data types for methods with NjsStringLike args.

diffstat:

test/ts/test.ts | 5 ++++-
ts/ngx_http_js_module.d.ts | 20 ++++++++++----------
ts/ngx_stream_js_module.d.ts | 8 ++++----
ts/njs_core.d.ts | 1 +
ts/njs_modules/crypto.d.ts | 6 +++---
ts/njs_modules/fs.d.ts | 8 ++++----
6 files changed, 26 insertions(+), 22 deletions(-)

diffs (207 lines):

diff -r 434f20c29f4c -r 5bd78c74777a test/ts/test.ts
--- a/test/ts/test.ts Wed Nov 25 10:47:47 2020 +0000
+++ b/test/ts/test.ts Thu Nov 26 11:10:59 2020 +0000
@@ -49,6 +49,7 @@ function http_module(r: NginxHTTPRequest
// r.log

r.log(bs);
+ r.log(Buffer.from("abc"));
r.log(r.headersOut['Connection'] ?? '');

// r.variables
@@ -61,7 +62,7 @@ function http_module(r: NginxHTTPRequest
r.subrequest('/p/sub2', {method:'POST'}).then(reply => r.return(reply.status));
vod = r.subrequest('/p/sub3', reply => r.return(reply.status));
vod = r.subrequest('/p/sub4', {method:'POST'}, reply => r.return(reply.status));
- vod = r.subrequest('/p/sub5', {detached:true});
+ vod = r.subrequest(Buffer.from('/p/sub5'), {detached:true});
// Warning: vod = r.subrequest('/p/sub9', {detached:true}, reply => r.return(reply.status));
r.subrequest('/p/sub6', 'a=1&b=2').then(reply => r.return(reply.status,
JSON.stringify(JSON.parse(reply.responseBody ?? ''))));
@@ -73,6 +74,8 @@ function fs_module() {

s = fs.readFileSync('/path', 'utf8');
s = fs.readFileSync(Buffer.from('/path'), {encoding:'hex'});
+
+ fs.writeFileSync('/path', Buffer.from('abc'));
}

function qs_module(str: NjsByteString) {
diff -r 434f20c29f4c -r 5bd78c74777a ts/ngx_http_js_module.d.ts
--- a/ts/ngx_http_js_module.d.ts Wed Nov 25 10:47:47 2020 +0000
+++ b/ts/ngx_http_js_module.d.ts Thu Nov 26 11:10:59 2020 +0000
@@ -263,7 +263,7 @@ interface NginxHTTPRequest {
* Writes a string to the error log on the error level of logging.
* @param message Message to log.
*/
- error(message: NjsStringLike): void;
+ error(message: NjsStringOrBuffer): void;
/**
* Finishes sending a response to the client.
*/
@@ -286,12 +286,12 @@ interface NginxHTTPRequest {
* The actual redirect happens after the handler execution is completed.
* @param uri Location to redirect to.
*/
- internalRedirect(uri: NjsStringLike): void;
+ internalRedirect(uri: NjsStringOrBuffer): void;
/**
* Writes a string to the error log on the info level of logging.
* @param message Message to log.
*/
- log(message: NjsStringLike): void;
+ log(message: NjsStringOrBuffer): void;
/**
* HTTP method.
*/
@@ -323,11 +323,11 @@ interface NginxHTTPRequest {
* @param status Respose status code.
* @param body Respose body.
*/
- return(status: number, body?: NjsStringLike): void;
+ return(status: number, body?: NjsStringOrBuffer): void;
/**
* Sends the HTTP headers to the client.
*/
- send(part: NjsStringLike): void;
+ send(part: NjsStringOrBuffer): void;
/**
* Sends the HTTP headers to the client.
*/
@@ -346,11 +346,11 @@ interface NginxHTTPRequest {
* @param options Subrequest options.
* @param callback Completion callback.
*/
- subrequest(uri: NjsStringLike, options: NginxSubrequestOptions & { detached: true }): void;
- subrequest(uri: NjsStringLike, options?: NginxSubrequestOptions | string): Promise<NginxHTTPRequest>;
- subrequest(uri: NjsStringLike, options: NginxSubrequestOptions & { detached?: false } | string,
+ subrequest(uri: NjsStringOrBuffer, options: NginxSubrequestOptions & { detached: true }): void;
+ subrequest(uri: NjsStringOrBuffer, options?: NginxSubrequestOptions | string): Promise<NginxHTTPRequest>;
+ subrequest(uri: NjsStringOrBuffer, options: NginxSubrequestOptions & { detached?: false } | string,
callback:(reply:NginxHTTPRequest) => void): void;
- subrequest(uri: NjsStringLike, callback:(reply:NginxHTTPRequest) => void): void;
+ subrequest(uri: NjsStringOrBuffer, callback:(reply:NginxHTTPRequest) => void): void;
/**
* Current URI in request, normalized.
*/
@@ -363,5 +363,5 @@ interface NginxHTTPRequest {
* Writes a string to the error log on the warn level of logging.
* @param message Message to log.
*/
- warn(message: NjsStringLike): void;
+ warn(message: NjsStringOrBuffer): void;
}
diff -r 434f20c29f4c -r 5bd78c74777a ts/ngx_stream_js_module.d.ts
--- a/ts/ngx_stream_js_module.d.ts Wed Nov 25 10:47:47 2020 +0000
+++ b/ts/ngx_stream_js_module.d.ts Thu Nov 26 11:10:59 2020 +0000
@@ -110,12 +110,12 @@ interface NginxStreamRequest {
* Writes a string to the error log on the error level of logging.
* @param message Message to log.
*/
- error(message: NjsStringLike): void;
+ error(message: NjsStringOrBuffer): void;
/**
* Writes a string to the error log on the info level of logging.
* @param message Message to log.
*/
- log(message: NjsStringLike): void;
+ log(message: NjsStringOrBuffer): void;
/**
* Unregisters the callback set by on() method.
*/
@@ -135,7 +135,7 @@ interface NginxStreamRequest {
* @param options Object used to override nginx buffer flags derived from
* an incoming data chunk buffer.
*/
- send(data: NjsStringLike, options?: NginxStreamSendOptions): void;
+ send(data: NjsStringOrBuffer, options?: NginxStreamSendOptions): void;
/**
* nginx variables object.
*/
@@ -144,5 +144,5 @@ interface NginxStreamRequest {
* Writes a string to the error log on the warn level of logging.
* @param message Message to log.
*/
- warn(message: NjsStringLike): void;
+ warn(message: NjsStringOrBuffer): void;
}
diff -r 434f20c29f4c -r 5bd78c74777a ts/njs_core.d.ts
--- a/ts/njs_core.d.ts Wed Nov 25 10:47:47 2020 +0000
+++ b/ts/njs_core.d.ts Thu Nov 26 11:10:59 2020 +0000
@@ -584,6 +584,7 @@ declare class Buffer extends Uint8Array
writeFloatLE(value: number, offset?: number): number;
}

+type NjsStringOrBuffer = NjsStringLike | Buffer | DataView | TypedArray;

// Global objects

diff -r 434f20c29f4c -r 5bd78c74777a ts/njs_modules/crypto.d.ts
--- a/ts/njs_modules/crypto.d.ts Wed Nov 25 10:47:47 2020 +0000
+++ b/ts/njs_modules/crypto.d.ts Thu Nov 26 11:10:59 2020 +0000
@@ -10,7 +10,7 @@ declare module "crypto" {
/**
* Updates the hash content with the given `data` and returns self.
*/
- update(data: NjsStringLike | Buffer | DataView | TypedArray): Hash;
+ update(data: NjsStringOrBuffer): Hash;

/**
* Calculates the digest of all of the data passed using `hash.update()`.
@@ -31,7 +31,7 @@ declare module "crypto" {
/**
* Updates the HMAC content with the given `data` and returns self.
*/
- update(data: NjsStringLike | Buffer | DataView | TypedArray): Hmac;
+ update(data: NjsStringOrBuffer): Hmac;

/**
* Calculates the HMAC digest of all of the data passed using `hmac.update()`.
@@ -65,7 +65,7 @@ declare module "crypto" {
* @param key The secret key.
* @returns An `HMAC` object.
*/
- createHmac(algorithm: Algorithm, key: NjsStringLike): Hmac;
+ createHmac(algorithm: Algorithm, key: NjsStringOrBuffer): Hmac;
}

const crypto: Crypto;
diff -r 434f20c29f4c -r 5bd78c74777a ts/njs_modules/fs.d.ts
--- a/ts/njs_modules/fs.d.ts Wed Nov 25 10:47:47 2020 +0000
+++ b/ts/njs_modules/fs.d.ts Thu Nov 26 11:10:59 2020 +0000
@@ -124,7 +124,7 @@ declare module "fs" {
* If `mode` is not supplied, the default of `0o666` is used.
* If `flag` is not supplied, the default of `'a'` is used.
*/
- appendFile(path: PathLike, data: NjsStringLike | Buffer, options?: WriteFileOptions): Promise<void>;
+ appendFile(path: PathLike, data: NjsStringOrBuffer, options?: WriteFileOptions): Promise<void>;

/**
* Asynchronously creates a directory at the specified `path`.
@@ -219,7 +219,7 @@ declare module "fs" {
* If `mode` is not supplied, the default of `0o666` is used.
* If `flag` is not supplied, the default of `'w'` is used.
*/
- writeFile(path: PathLike, data: NjsStringLike | Buffer, options?: WriteFileOptions): Promise<void>;
+ writeFile(path: PathLike, data: NjsStringOrBuffer, options?: WriteFileOptions): Promise<void>;
}

interface NjsFS {
@@ -264,7 +264,7 @@ declare module "fs" {
* If `mode` is not supplied, the default of `0o666` is used.
* If `flag` is not supplied, the default of `'a'` is used.
*/
- appendFileSync(path: PathLike, data: NjsStringLike | Buffer, options?: WriteFileOptions): void;
+ appendFileSync(path: PathLike, data: NjsStringOrBuffer, options?: WriteFileOptions): void;

/**
* Synchronously creates a directory at the specified `path`.
@@ -373,7 +373,7 @@ declare module "fs" {
* If `mode` is not supplied, the default of `0o666` is used.
* If `flag` is not supplied, the default of `'w'` is used.
*/
- writeFileSync(path: PathLike, data: NjsStringLike | Buffer, options?: WriteFileOptions): void;
+ writeFileSync(path: PathLike, data: NjsStringOrBuffer, options?: WriteFileOptions): void;
}

const fs: NjsFS;
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Thu, 26 Nov 2020 06:14:13 -0500
http://www.ldmicj.icu/read.php?29,290059,290059#msg-290059 Help - Nginx Azure Auth (10 replies) http://www.ldmicj.icu/read.php?29,290059,290059#msg-290059 I have configured nginx to authenticate with azure AD for login.

When I access the site abc.example.com it redirects to Azure for
authentication and redirects me back once the authentication is complete.

How ever when I try to access the site with https abc.example.com it does
not redirect for authentication.

Is there anyway I can get both http and https to redirect for azure auth.

Regards,
Pavan
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Pavan P Nginx Development Thu, 26 Nov 2020 01:22:06 -0500
http://www.ldmicj.icu/read.php?29,290054,290054#msg-290054 [njs] Modules: introduced Buffer alternatives for object properties. (no replies) http://www.ldmicj.icu/read.php?29,290054,290054#msg-290054 branches:
changeset: 1571:434f20c29f4c
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Wed Nov 25 10:47:47 2020 +0000
description:
Modules: introduced Buffer alternatives for object properties.

Buffer variant returns the property bytes as is, whereas the string
version may convert bytes invalid in UTF-8 encoding into replacement
character.

HTTP
new request object properties:
r.reqBody (r.requestBody),
r.resBody (r.responseBody),
r.vars (r.variables).

Stream
new stream object properties:
s.vars (s.variables).

new events:
The events' callbacks are identical to the string counterparts, except the data
argument:
upstream (upload),
downstream (download).

diffstat:

nginx/ngx_http_js_module.c | 71 +++++++++++++++++++++++--
nginx/ngx_js.h | 10 +++
nginx/ngx_stream_js_module.c | 117 ++++++++++++++++++++++++++++++++----------
src/njs.h | 2 +
src/njs_extern.c | 1 +
src/njs_value.c | 8 ++
src/njs_value.h | 1 +
7 files changed, 175 insertions(+), 35 deletions(-)

diffs (517 lines):

diff -r 0684301385d9 -r 434f20c29f4c nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c Wed Nov 25 10:47:25 2020 +0000
+++ b/nginx/ngx_http_js_module.c Wed Nov 25 10:47:47 2020 +0000
@@ -43,6 +43,7 @@ typedef struct {
ngx_int_t status;
njs_opaque_value_t request;
njs_opaque_value_t request_body;
+ njs_opaque_value_t response_body;
ngx_str_t redirect_uri;
ngx_array_t promise_callbacks;
} ngx_http_js_ctx_t;
@@ -319,9 +320,19 @@ static njs_external_t ngx_http_js_ext_r
{
.flags = NJS_EXTERN_PROPERTY,
.name.string = njs_str("requestBody"),
+ .u.property = {
+ .handler = ngx_http_js_ext_get_request_body,
+ .magic32 = NGX_JS_STRING,
+ }
+ },
+
+ {
+ .flags = NJS_EXTERN_PROPERTY,
+ .name.string = njs_str("reqBody"),
.enumerable = 1,
.u.property = {
.handler = ngx_http_js_ext_get_request_body,
+ .magic32 = NGX_JS_BUFFER,
}
},

@@ -336,9 +347,19 @@ static njs_external_t ngx_http_js_ext_r
{
.flags = NJS_EXTERN_PROPERTY,
.name.string = njs_str("responseBody"),
+ .u.property = {
+ .handler = ngx_http_js_ext_get_response_body,
+ .magic32 = NGX_JS_STRING,
+ }
+ },
+
+ {
+ .flags = NJS_EXTERN_PROPERTY,
+ .name.string = njs_str("resBody"),
.enumerable = 1,
.u.property = {
.handler = ngx_http_js_ext_get_response_body,
+ .magic32 = NGX_JS_BUFFER,
}
},

@@ -379,6 +400,17 @@ static njs_external_t ngx_http_js_ext_r
.u.object = {
.writable = 1,
.prop_handler = ngx_http_js_ext_variables,
+ .magic32 = NGX_JS_STRING,
+ }
+ },
+
+ {
+ .flags = NJS_EXTERN_OBJECT,
+ .name.string = njs_str("vars"),
+ .u.object = {
+ .writable = 1,
+ .prop_handler = ngx_http_js_ext_variables,
+ .magic32 = NGX_JS_BUFFER,
}
},

@@ -1891,8 +1923,12 @@ ngx_http_js_ext_get_request_body(njs_vm_
request_body = (njs_value_t *) &ctx->request_body;

if (!njs_value_is_null(request_body)) {
- njs_value_assign(retval, request_body);
- return NJS_OK;
+ if ((njs_vm_prop_magic32(prop) == NGX_JS_BUFFER)
+ == (uint32_t) njs_value_is_buffer(request_body))
+ {
+ njs_value_assign(retval, request_body);
+ return NJS_OK;
+ }
}

if (r->request_body == NULL || r->request_body->bufs == NULL) {
@@ -1939,8 +1975,7 @@ ngx_http_js_ext_get_request_body(njs_vm_

done:

- ret = njs_vm_value_string_set(vm, request_body, body, len);
-
+ ret = ngx_js_prop(vm, njs_vm_prop_magic32(prop), request_body, body, len);
if (ret != NJS_OK) {
return NJS_ERROR;
}
@@ -2220,7 +2255,8 @@ ngx_http_js_ext_variables(njs_vm_t *vm,
return NJS_DECLINED;
}

- return njs_vm_value_string_set(vm, retval, vv->data, vv->len);
+ return ngx_js_prop(vm, njs_vm_prop_magic32(prop), retval, vv->data,
+ vv->len);
}

cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
@@ -2743,7 +2779,10 @@ ngx_http_js_ext_get_response_body(njs_vm
{
size_t len;
u_char *p;
+ njs_int_t ret;
ngx_buf_t *b;
+ njs_value_t *response_body;
+ ngx_http_js_ctx_t *ctx;
ngx_http_request_t *r;

r = njs_vm_external(vm, value);
@@ -2752,6 +2791,18 @@ ngx_http_js_ext_get_response_body(njs_vm
return NJS_DECLINED;
}

+ ctx = ngx_http_get_module_ctx(r, ngx_http_js_module);
+ response_body = (njs_value_t *) &ctx->response_body;
+
+ if (!njs_value_is_null(response_body)) {
+ if ((njs_vm_prop_magic32(prop) == NGX_JS_BUFFER)
+ == (uint32_t) njs_value_is_buffer(response_body))
+ {
+ njs_value_assign(retval, response_body);
+ return NJS_OK;
+ }
+ }
+
b = r->out ? r->out->buf : NULL;

if (b == NULL) {
@@ -2761,8 +2812,9 @@ ngx_http_js_ext_get_response_body(njs_vm

len = b->last - b->pos;

- p = njs_vm_value_string_alloc(vm, retval, len);
+ p = ngx_pnalloc(r->pool, len);
if (p == NULL) {
+ njs_vm_memory_error(vm);
return NJS_ERROR;
}

@@ -2770,6 +2822,13 @@ ngx_http_js_ext_get_response_body(njs_vm
ngx_memcpy(p, b->pos, len);
}

+ ret = ngx_js_prop(vm, njs_vm_prop_magic32(prop), response_body, p, len);
+ if (ret != NJS_OK) {
+ return NJS_ERROR;
+ }
+
+ njs_value_assign(retval, response_body);
+
return NJS_OK;
}

diff -r 0684301385d9 -r 434f20c29f4c nginx/ngx_js.h
--- a/nginx/ngx_js.h Wed Nov 25 10:47:25 2020 +0000
+++ b/nginx/ngx_js.h Wed Nov 25 10:47:47 2020 +0000
@@ -15,10 +15,20 @@
#include <njs.h>


+#define NGX_JS_UNSET 0
+#define NGX_JS_STRING 1
+#define NGX_JS_BUFFER 2
+
+
#define ngx_external_connection(vm, ext) \
(*((ngx_connection_t **) ((u_char *) ext + njs_vm_meta(vm, 0))))


+#define ngx_js_prop(vm, type, value, start, len) \
+ ((type == NGX_JS_STRING) ? njs_vm_value_string_set(vm, value, start, len) \
+ : njs_vm_value_buffer_set(vm, value, start, len))
+
+
ngx_int_t ngx_js_call(njs_vm_t *vm, ngx_str_t *s, njs_opaque_value_t *value,
ngx_log_t *log);

diff -r 0684301385d9 -r 434f20c29f4c nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c Wed Nov 25 10:47:25 2020 +0000
+++ b/nginx/ngx_stream_js_module.c Wed Nov 25 10:47:47 2020 +0000
@@ -39,6 +39,12 @@ typedef struct {


typedef struct {
+ njs_vm_event_t ev;
+ ngx_uint_t data_type;
+} ngx_stream_js_ev_t;
+
+
+typedef struct {
njs_vm_t *vm;
ngx_log_t *log;
njs_opaque_value_t args[3];
@@ -51,7 +57,7 @@ typedef struct {
#define NGX_JS_EVENT_UPLOAD 0
#define NGX_JS_EVENT_DOWNLOAD 1
#define NGX_JS_EVENT_MAX 2
- njs_vm_event_t events[2];
+ ngx_stream_js_ev_t events[2];
unsigned from_upstream:1;
unsigned filter:1;
unsigned in_progress:1;
@@ -79,7 +85,7 @@ static void ngx_stream_js_drop_events(ng
static void ngx_stream_js_cleanup_ctx(void *data);
static void ngx_stream_js_cleanup_vm(void *data);
static njs_int_t ngx_stream_js_buffer_arg(ngx_stream_session_t *s,
- njs_value_t *buffer);
+ njs_value_t *buffer, ngx_uint_t data_type);
static njs_int_t ngx_stream_js_flags_arg(ngx_stream_session_t *s,
njs_value_t *flags);
static njs_vm_event_t *ngx_stream_js_event(ngx_stream_session_t *s,
@@ -233,6 +239,17 @@ static njs_external_t ngx_stream_js_ext
.u.object = {
.writable = 1,
.prop_handler = ngx_stream_js_ext_variables,
+ .magic32 = NGX_JS_STRING,
+ }
+ },
+
+ {
+ .flags = NJS_EXTERN_OBJECT,
+ .name.string = njs_str("vars"),
+ .u.object = {
+ .writable = 1,
+ .prop_handler = ngx_stream_js_ext_variables,
+ .magic32 = NGX_JS_BUFFER,
}
},

@@ -412,6 +429,7 @@ ngx_stream_js_phase_handler(ngx_stream_s
njs_int_t ret;
ngx_int_t rc;
ngx_connection_t *c;
+ ngx_stream_js_ev_t *event;
ngx_stream_js_ctx_t *ctx;

if (name->len == 0) {
@@ -444,8 +462,11 @@ ngx_stream_js_phase_handler(ngx_stream_s
}
}

- if (ctx->events[NGX_JS_EVENT_UPLOAD] != NULL) {
- ret = ngx_stream_js_buffer_arg(s, njs_value_arg(&ctx->args[1]));
+ event = &ctx->events[NGX_JS_EVENT_UPLOAD];
+
+ if (event->ev != NULL) {
+ ret = ngx_stream_js_buffer_arg(s, njs_value_arg(&ctx->args[1]),
+ event->data_type);
if (ret != NJS_OK) {
goto exception;
}
@@ -455,8 +476,7 @@ ngx_stream_js_phase_handler(ngx_stream_s
goto exception;
}

- njs_vm_post_event(ctx->vm, ctx->events[NGX_JS_EVENT_UPLOAD],
- njs_value_arg(&ctx->args[1]), 2);
+ njs_vm_post_event(ctx->vm, event->ev, njs_value_arg(&ctx->args[1]), 2);

rc = njs_vm_run(ctx->vm);
if (rc == NJS_ERROR) {
@@ -466,7 +486,7 @@ ngx_stream_js_phase_handler(ngx_stream_s

if (njs_vm_pending(ctx->vm)) {
ctx->in_progress = 1;
- rc = ctx->events[NGX_JS_EVENT_UPLOAD] ? NGX_AGAIN : NGX_DONE;
+ rc = ctx->events[NGX_JS_EVENT_UPLOAD].ev ? NGX_AGAIN : NGX_DONE;

} else {
ctx->in_progress = 0;
@@ -490,8 +510,8 @@ exception:


#define ngx_stream_event(from_upstream) \
- (from_upstream ? ctx->events[NGX_JS_EVENT_DOWNLOAD] \
- : ctx->events[NGX_JS_EVENT_UPLOAD])
+ (from_upstream ? &ctx->events[NGX_JS_EVENT_DOWNLOAD] \
+ : &ctx->events[NGX_JS_EVENT_UPLOAD])


static ngx_int_t
@@ -503,6 +523,7 @@ ngx_stream_js_body_filter(ngx_stream_ses
ngx_int_t rc;
ngx_chain_t *out, *cl;
ngx_connection_t *c;
+ ngx_stream_js_ev_t *event;
ngx_stream_js_ctx_t *ctx;
ngx_stream_js_srv_conf_t *jscf;

@@ -543,8 +564,11 @@ ngx_stream_js_body_filter(ngx_stream_ses
while (in) {
ctx->buf = in->buf;

- if (ngx_stream_event(from_upstream) != NULL) {
- ret = ngx_stream_js_buffer_arg(s, njs_value_arg(&ctx->args[1]));
+ event = ngx_stream_event(from_upstream);
+
+ if (event->ev != NULL) {
+ ret = ngx_stream_js_buffer_arg(s, njs_value_arg(&ctx->args[1]),
+ event->data_type);
if (ret != NJS_OK) {
goto exception;
}
@@ -554,7 +578,7 @@ ngx_stream_js_body_filter(ngx_stream_ses
goto exception;
}

- njs_vm_post_event(ctx->vm, ngx_stream_event(from_upstream),
+ njs_vm_post_event(ctx->vm, event->ev,
njs_value_arg(&ctx->args[1]), 2);

rc = njs_vm_run(ctx->vm);
@@ -729,9 +753,9 @@ ngx_stream_js_drop_events(ngx_stream_js_
ngx_uint_t i;

for (i = 0; i < NGX_JS_EVENT_MAX; i++) {
- if (ctx->events[i] != NULL) {
- njs_vm_del_event(ctx->vm, ctx->events[i]);
- ctx->events[i] = NULL;
+ if (ctx->events[i].ev != NULL) {
+ njs_vm_del_event(ctx->vm, ctx->events[i].ev);
+ ctx->events[i].ev = NULL;
}
}
}
@@ -762,7 +786,8 @@ ngx_stream_js_cleanup_vm(void *data)


static njs_int_t
-ngx_stream_js_buffer_arg(ngx_stream_session_t *s, njs_value_t *buffer)
+ngx_stream_js_buffer_arg(ngx_stream_session_t *s, njs_value_t *buffer,
+ ngx_uint_t data_type)
{
size_t len;
u_char *p;
@@ -777,8 +802,9 @@ ngx_stream_js_buffer_arg(ngx_stream_sess

len = b ? b->last - b->pos : 0;

- p = njs_vm_value_string_alloc(ctx->vm, buffer, len);
+ p = ngx_pnalloc(c->pool, len);
if (p == NULL) {
+ njs_vm_memory_error(ctx->vm);
return NJS_ERROR;
}

@@ -786,11 +812,10 @@ ngx_stream_js_buffer_arg(ngx_stream_sess
ngx_memcpy(p, b->pos, len);
}

- return NJS_OK;
+ return ngx_js_prop(ctx->vm, data_type, buffer, p, len);
}


-
static njs_int_t
ngx_stream_js_flags_arg(ngx_stream_session_t *s, njs_value_t *flags)
{
@@ -821,12 +846,37 @@ ngx_stream_js_flags_arg(ngx_stream_sessi
static njs_vm_event_t *
ngx_stream_js_event(ngx_stream_session_t *s, njs_str_t *event)
{
- ngx_uint_t i, n;
+ ngx_uint_t i, n, type;
ngx_stream_js_ctx_t *ctx;

- static const njs_str_t events[] = {
- njs_str("upload"),
- njs_str("download")
+ static const struct {
+ ngx_str_t name;
+ ngx_uint_t data_type;
+ ngx_uint_t id;
+ } events[] = {
+ {
+ ngx_string("upload"),
+ NGX_JS_STRING,
+ NGX_JS_EVENT_UPLOAD,
+ },
+
+ {
+ ngx_string("download"),
+ NGX_JS_STRING,
+ NGX_JS_EVENT_DOWNLOAD,
+ },
+
+ {
+ ngx_string("upstream"),
+ NGX_JS_BUFFER,
+ NGX_JS_EVENT_UPLOAD,
+ },
+
+ {
+ ngx_string("downstream"),
+ NGX_JS_BUFFER,
+ NGX_JS_EVENT_DOWNLOAD,
+ },
};

ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
@@ -835,8 +885,9 @@ ngx_stream_js_event(ngx_stream_session_t
n = sizeof(events) / sizeof(events[0]);

while (i < n) {
- if (event->length == events[i].length
- && ngx_memcmp(event->start, events[i].start, event->length) == 0)
+ if (event->length == events[i].name.len
+ && ngx_memcmp(event->start, events[i].name.data, event->length)
+ == 0)
{
break;
}
@@ -849,11 +900,18 @@ ngx_stream_js_event(ngx_stream_session_t
return NULL;
}

- if (i == 0) {
- return &ctx->events[NGX_JS_EVENT_UPLOAD];
+ ctx->events[events[i].id].data_type = events[i].data_type;
+
+ for (n = 0; n < NGX_JS_EVENT_MAX; n++) {
+ type = ctx->events[n].data_type;
+ if (type != NGX_JS_UNSET && type != events[i].data_type) {
+ njs_vm_error(ctx->vm, "mixing string and buffer events"
+ " is not allowed");
+ return NULL;
+ }
}

- return &ctx->events[NGX_JS_EVENT_DOWNLOAD];
+ return &ctx->events[events[i].id].ev;
}


@@ -1131,7 +1189,8 @@ ngx_stream_js_ext_variables(njs_vm_t *vm
return NJS_DECLINED;
}

- return njs_vm_value_string_set(vm, retval, vv->data, vv->len);
+ return ngx_js_prop(vm, njs_vm_prop_magic32(prop), retval, vv->data,
+ vv->len);
}

cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
diff -r 0684301385d9 -r 434f20c29f4c src/njs.h
--- a/src/njs.h Wed Nov 25 10:47:25 2020 +0000
+++ b/src/njs.h Wed Nov 25 10:47:47 2020 +0000
@@ -144,6 +144,7 @@ struct njs_external_s {
unsigned configurable;
unsigned enumerable;
njs_prop_handler_t prop_handler;
+ uint32_t magic32;
njs_exotic_keys_t keys;
} object;
} u;
@@ -389,6 +390,7 @@ NJS_EXPORT njs_int_t njs_value_is_string
NJS_EXPORT njs_int_t njs_value_is_object(const njs_value_t *value);
NJS_EXPORT njs_int_t njs_value_is_array(const njs_value_t *value);
NJS_EXPORT njs_int_t njs_value_is_function(const njs_value_t *value);
+NJS_EXPORT njs_int_t njs_value_is_buffer(const njs_value_t *value);

NJS_EXPORT njs_int_t njs_vm_object_alloc(njs_vm_t *vm, njs_value_t *retval,
...);
diff -r 0684301385d9 -r 434f20c29f4c src/njs_extern.c
--- a/src/njs_extern.c Wed Nov 25 10:47:25 2020 +0000
+++ b/src/njs_extern.c Wed Nov 25 10:47:47 2020 +0000
@@ -132,6 +132,7 @@ njs_external_add(njs_vm_t *vm, njs_arr_t
next->configurable = external->u.object.configurable;
next->enumerable = external->u.object.enumerable;
next->prop_handler = external->u.object.prop_handler;
+ next->magic32 = external->u.object.magic32;
next->keys = external->u.object.keys;

break;
diff -r 0684301385d9 -r 434f20c29f4c src/njs_value.c
--- a/src/njs_value.c Wed Nov 25 10:47:25 2020 +0000
+++ b/src/njs_value.c Wed Nov 25 10:47:47 2020 +0000
@@ -492,6 +492,13 @@ njs_value_is_function(const njs_value_t
}


+njs_int_t
+njs_value_is_buffer(const njs_value_t *value)
+{
+ return njs_is_typed_array(value);
+}
+
+
/*
* ES5.1, 8.12.1: [[GetOwnProperty]], [[GetProperty]].
* The njs_property_query() returns values
@@ -932,6 +939,7 @@ njs_external_property_query(njs_vm_t *vm
* njs_set_null(&prop->setter);
*/

+ prop->value.data.magic32 = slots->magic32;
prop->name = pq->key;

pq->lhq.value = prop;
diff -r 0684301385d9 -r 434f20c29f4c src/njs_value.h
--- a/src/njs_value.h Wed Nov 25 10:47:25 2020 +0000
+++ b/src/njs_value.h Wed Nov 25 10:47:47 2020 +0000
@@ -189,6 +189,7 @@ union njs_value_s {
typedef struct {
/* Get, also Set if writable, also Delete if configurable. */
njs_prop_handler_t prop_handler;
+ uint32_t magic32;
unsigned writable:1;
unsigned configurable:1;
unsigned enumerable:1;
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Wed, 25 Nov 2020 06:10:03 -0500
http://www.ldmicj.icu/read.php?29,290053,290053#msg-290053 [njs] Stream: improved vm events handling. (no replies) http://www.ldmicj.icu/read.php?29,290053,290053#msg-290053 branches:
changeset: 1570:0684301385d9
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Wed Nov 25 10:47:25 2020 +0000
description:
Stream: improved vm events handling.

diffstat:

nginx/ngx_stream_js_module.c | 54 ++++++++++++++++++++++---------------------
1 files changed, 28 insertions(+), 26 deletions(-)

diffs (126 lines):

diff -r e51da8c71f26 -r 0684301385d9 nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c Fri Nov 20 12:29:30 2020 +0300
+++ b/nginx/ngx_stream_js_module.c Wed Nov 25 10:47:25 2020 +0000
@@ -48,8 +48,10 @@ typedef struct {
ngx_chain_t *busy;
ngx_stream_session_t *session;
ngx_int_t status;
- njs_vm_event_t upload_event;
- njs_vm_event_t download_event;
+#define NGX_JS_EVENT_UPLOAD 0
+#define NGX_JS_EVENT_DOWNLOAD 1
+#define NGX_JS_EVENT_MAX 2
+ njs_vm_event_t events[2];
unsigned from_upstream:1;
unsigned filter:1;
unsigned in_progress:1;
@@ -73,6 +75,7 @@ static ngx_int_t ngx_stream_js_body_filt
static ngx_int_t ngx_stream_js_variable(ngx_stream_session_t *s,
ngx_stream_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_stream_js_init_vm(ngx_stream_session_t *s);
+static void ngx_stream_js_drop_events(ngx_stream_js_ctx_t *ctx);
static void ngx_stream_js_cleanup_ctx(void *data);
static void ngx_stream_js_cleanup_vm(void *data);
static njs_int_t ngx_stream_js_buffer_arg(ngx_stream_session_t *s,
@@ -441,7 +444,7 @@ ngx_stream_js_phase_handler(ngx_stream_s
}
}

- if (ctx->upload_event != NULL) {
+ if (ctx->events[NGX_JS_EVENT_UPLOAD] != NULL) {
ret = ngx_stream_js_buffer_arg(s, njs_value_arg(&ctx->args[1]));
if (ret != NJS_OK) {
goto exception;
@@ -452,7 +455,7 @@ ngx_stream_js_phase_handler(ngx_stream_s
goto exception;
}

- njs_vm_post_event(ctx->vm, ctx->upload_event,
+ njs_vm_post_event(ctx->vm, ctx->events[NGX_JS_EVENT_UPLOAD],
njs_value_arg(&ctx->args[1]), 2);

rc = njs_vm_run(ctx->vm);
@@ -463,7 +466,7 @@ ngx_stream_js_phase_handler(ngx_stream_s

if (njs_vm_pending(ctx->vm)) {
ctx->in_progress = 1;
- rc = ctx->upload_event ? NGX_AGAIN : NGX_DONE;
+ rc = ctx->events[NGX_JS_EVENT_UPLOAD] ? NGX_AGAIN : NGX_DONE;

} else {
ctx->in_progress = 0;
@@ -487,7 +490,8 @@ exception:


#define ngx_stream_event(from_upstream) \
- (from_upstream ? ctx->download_event : ctx->upload_event)
+ (from_upstream ? ctx->events[NGX_JS_EVENT_DOWNLOAD] \
+ : ctx->events[NGX_JS_EVENT_UPLOAD])


static ngx_int_t
@@ -720,19 +724,25 @@ ngx_stream_js_init_vm(ngx_stream_session


static void
+ngx_stream_js_drop_events(ngx_stream_js_ctx_t *ctx)
+{
+ ngx_uint_t i;
+
+ for (i = 0; i < NGX_JS_EVENT_MAX; i++) {
+ if (ctx->events[i] != NULL) {
+ njs_vm_del_event(ctx->vm, ctx->events[i]);
+ ctx->events[i] = NULL;
+ }
+ }
+}
+
+
+static void
ngx_stream_js_cleanup_ctx(void *data)
{
ngx_stream_js_ctx_t *ctx = data;

- if (ctx->upload_event != NULL) {
- njs_vm_del_event(ctx->vm, ctx->upload_event);
- ctx->upload_event = NULL;
- }
-
- if (ctx->download_event != NULL) {
- njs_vm_del_event(ctx->vm, ctx->download_event);
- ctx->download_event = NULL;
- }
+ ngx_stream_js_drop_events(ctx);

if (njs_vm_pending(ctx->vm)) {
ngx_log_error(NGX_LOG_ERR, ctx->log, 0, "pending events");
@@ -840,10 +850,10 @@ ngx_stream_js_event(ngx_stream_session_t
}

if (i == 0) {
- return &ctx->upload_event;
+ return &ctx->events[NGX_JS_EVENT_UPLOAD];
}

- return &ctx->download_event;
+ return &ctx->events[NGX_JS_EVENT_DOWNLOAD];
}


@@ -911,15 +921,7 @@ ngx_stream_js_ext_done(njs_vm_t *vm, njs

ctx->status = status;

- if (ctx->upload_event != NULL) {
- njs_vm_del_event(ctx->vm, ctx->upload_event);
- ctx->upload_event = NULL;
- }
-
- if (ctx->download_event != NULL) {
- njs_vm_del_event(ctx->vm, ctx->download_event);
- ctx->download_event = NULL;
- }
+ ngx_stream_js_drop_events(ctx);

njs_value_undefined_set(njs_vm_retval(vm));

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Wed, 25 Nov 2020 06:10:03 -0500
http://www.ldmicj.icu/read.php?29,290040,290040#msg-290040 [nginx] nginx-1.19.5-RELEASE (no replies) http://www.ldmicj.icu/read.php?29,290040,290040#msg-290040 branches:
changeset: 7747:8e5b068f761c
user: Maxim Dounin <mdounin@mdounin.ru>
date: Tue Nov 24 18:06:34 2020 +0300
description:
nginx-1.19.5-RELEASE

diffstat:

docs/xml/nginx/changes.xml | 56 ++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 56 insertions(+), 0 deletions(-)

diffs (66 lines):

diff -r 88eca63261c3 -r 8e5b068f761c docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml Wed Nov 18 18:41:16 2020 -0800
+++ b/docs/xml/nginx/changes.xml Tue Nov 24 18:06:34 2020 +0300
@@ -5,6 +5,62 @@
<change_log title="nginx">


+<changes ver="1.19.5" date="2020-11-24">
+
+<change type="feature">
+<para lang="ru">
+ключ -e.
+</para>
+<para lang="en">
+the -e switch.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+при сборке дополнительных модулей
+теперь можно указывать одни и те же исходные файлы в разных модулях.
+</para>
+<para lang="en">
+the same source files can now be specified in different modules
+while building addon modules.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+SSL shutdown не работал
+при закрытии соединений с ожиданием дополнительных данных (lingering close).
+</para>
+<para lang="en">
+SSL shutdown did not work
+when lingering close was used.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+при работе с gRPC-бэкендами
+могли возникать ошибки "upstream sent frame for closed stream".
+</para>
+<para lang="en">
+"upstream sent frame for closed stream" errors might occur
+when working with gRPC backends.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+во внутреннем API для обработки тела запроса.
+</para>
+<para lang="en">
+in request body filters internal API.
+</para>
+</change>
+
+</changes>
+
+
<changes ver="1.19.4" date="2020-10-27">

<change type="feature">
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Maxim Dounin Nginx Development Tue, 24 Nov 2020 10:12:06 -0500
http://www.ldmicj.icu/read.php?29,290039,290039#msg-290039 [nginx] release-1.19.5 tag (no replies) http://www.ldmicj.icu/read.php?29,290039,290039#msg-290039 branches:
changeset: 7748:66a441bf669b
user: Maxim Dounin <mdounin@mdounin.ru>
date: Tue Nov 24 18:06:34 2020 +0300
description:
release-1.19.5 tag

diffstat:

.hgtags | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff -r 8e5b068f761c -r 66a441bf669b .hgtags
--- a/.hgtags Tue Nov 24 18:06:34 2020 +0300
+++ b/.hgtags Tue Nov 24 18:06:34 2020 +0300
@@ -454,3 +454,4 @@ 062920e2f3bf871ef7a3d8496edec1b3065faf80
a7b46539f507e6c64efa0efda69ad60b6f4ffbce release-1.19.2
3cbc2602325f0ac08917a4397d76f5155c34b7b1 release-1.19.3
dc0cc425fa63a80315f6efb68697cadb6626cdf2 release-1.19.4
+8e5b068f761cd512d10c9671fbde0b568c1fd08b release-1.19.5
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Maxim Dounin Nginx Development Tue, 24 Nov 2020 10:12:06 -0500
http://www.ldmicj.icu/read.php?29,290037,290037#msg-290037 [PATCH] [PATCH 1 of 8] Cloned liburing library (no replies) http://www.ldmicj.icu/read.php?29,290037,290037#msg-290037

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Tue, 24 Nov 2020 03:50:07 -0500
http://www.ldmicj.icu/read.php?29,290036,290036#msg-290036 [PATCH 8 of 8] new io_uring event module (2 replies) http://www.ldmicj.icu/read.php?29,290036,290036#msg-290036 # User SoYun Seong <thdbsdox12@gmail.com>
# Date 1606129415 0
# Mon Nov 23 11:03:35 2020 +0000
# Node ID d7420a5777b63c8a8cfb7e98a522893490995510
# Parent 09dfe4a92414513c6bd3c18d871e8a76ed19c3d7
new io_uring event module.

I implemented ngx_uring_module using Linux io_uring API to improve performance of Nginx for Linux by minimizing system calls. There are performance improvements in both request/sec and average latency. The result is located at https://github.com/dachshu/nginx.

However, there are some places that uses local variable buffer(stack) to recv and send data. To do asynchronous IO, recv and send buffers should be located at safe memory(like heap, data). Therefore it is needed to make these codes to use allocated memory from memory pool when using asynchronous IO.

Also I am working on improve performance of Nginx for Windows using Registered IO and IOCP.

diff -r 09dfe4a92414 -r d7420a5777b6 auto/os/linux
--- a/auto/os/linux Mon Nov 23 11:01:36 2020 +0000
+++ b/auto/os/linux Mon Nov 23 11:03:35 2020 +0000
@@ -89,6 +89,30 @@
fi


+# io_uring
+
+ngx_feature="uring"
+ngx_feature_name="NGX_HAVE_URING"
+ngx_feature_run=yes
+ngx_feature_incs="#include <liburing.h>"
+ngx_feature_path="-I src/liburing/src/include/"
+ngx_feature_libs="-L src/liburing/src/ -luring"
+ngx_feature_test="struct io_uring ring;
+ struct io_uring_params params;
+ if (io_uring_queue_init_params(32768, &ring, &params) < 0) return 1;
+ if (!(params.features & IORING_FEAT_FAST_POLL)) return 1;"
+. auto/feature
+
+if [ $ngx_found = yes ]; then
+ have=NGX_HAVE_CLEAR_EVENT . auto/have
+ CORE_SRCS="$CORE_SRCS $URING_SRCS"
+ CORE_INCS="$CORE_INCS $ngx_feature_path"
+ CORE_LIBS="$CORE_LIBS $ngx_feature_libs"
+ EVENT_MODULES="$EVENT_MODULES $URING_MODULE"
+ EVENT_FOUND=YES
+fi
+
+
# O_PATH and AT_EMPTY_PATH were introduced in 2.6.39, glibc 2.14

ngx_feature="O_PATH"
diff -r 09dfe4a92414 -r d7420a5777b6 auto/sources
--- a/auto/sources Mon Nov 23 11:01:36 2020 +0000
+++ b/auto/sources Mon Nov 23 11:03:35 2020 +0000
@@ -120,6 +120,9 @@
EPOLL_MODULE=ngx_epoll_module
EPOLL_SRCS=src/event/modules/ngx_epoll_module.c

+URING_MODULE=ngx_uring_module
+URING_SRCS=src/event/modules/ngx_uring_module.c
+
IOCP_MODULE=ngx_iocp_module
IOCP_SRCS=src/event/modules/ngx_iocp_module.c

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Wed, 25 Nov 2020 07:44:10 -0500
http://www.ldmicj.icu/read.php?29,290035,290035#msg-290035 [PATCH 4 of 8] Added io_uring event module (no replies) http://www.ldmicj.icu/read.php?29,290035,290035#msg-290035 # User SoYun Seong <thdbsdox12@gmail.com>
# Date 1606128722 0
# Mon Nov 23 10:52:02 2020 +0000
# Node ID d7f0f0b78a24724c362cd0306dc6bbed82c66e62
# Parent ce34097321cf3cb7f9de55075bd1788577bb1ad8
Added io_uring event module.

Implemented nginx io_uring module that processes event loop with io_uring API. It requires Linux kernel 5.7 or higher with IORING_FEAT_FAST_POLL. And it supports only tcp protocol yet.
To move data between a file and a socket without copying between kernel address space and user address space, I implemented asynchronous sendfile of io_uring module using two IORING_OP_SPLICE requests and pipes.
However, performance of the splice requests is getting poor after accepting a client. I'm finding the reason of this problem.
Because of this problem, I implemented sendfile of ngx_uring_module with a read request and a send request. This makes copy between kernel space and user space.
This sendfile implementation has a high overhead when sending a big size of file.
To alleviate this, when sending a file over page size(4KB), ngx_uring_module uses a Linux sendfile() system call instead.

diff -r ce34097321cf -r d7f0f0b78a24 src/event/modules/ngx_uring_module.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/src/event/modules/ngx_uring_module.c Mon Nov 23 10:52:02 2020 +0000
@@ -0,0 +1,1191 @@
+
+#include <ngx_config.h>
+#include <ngx_core.h>
+#include <ngx_event.h>
+
+#include "liburing.h"
+
+
+typedef struct {
+ ngx_uint_t entries;
+ size_t sendfile_bound;
+} ngx_uring_conf_t;
+
+
+typedef struct {
+ ngx_connection_t *conn;
+ ngx_uint_t ev;
+ void *buf;
+} ngx_uring_info_t;
+
+
+static ngx_int_t ngx_uring_init(ngx_cycle_t *cycle, ngx_msec_t timer);
+static void ngx_uring_done(ngx_cycle_t *cycle);
+static ngx_int_t ngx_uring_add_event(ngx_event_t *ev, ngx_int_t event,
+ ngx_uint_t flags);
+static ngx_int_t ngx_uring_add_connection(ngx_connection_t *c);
+static ngx_int_t ngx_uring_del_connection(ngx_connection_t *c,
+ ngx_uint_t flags);
+static ngx_int_t ngx_uring_process_events(ngx_cycle_t *cycle, ngx_msec_t timer,
+ ngx_uint_t flags);
+
+static void *ngx_uring_create_conf(ngx_cycle_t *cycle);
+static char *ngx_uring_init_conf(ngx_cycle_t *cycle, void *conf);
+
+
+ngx_int_t ngx_uring_accept(ngx_connection_t *c);
+ssize_t ngx_uring_recv(ngx_connection_t *c, u_char *buf, size_t size);
+ssize_t ngx_uring_readv_chain(ngx_connection_t *c, ngx_chain_t *chain, off_t limit);
+ssize_t ngx_uring_send(ngx_connection_t *c, u_char *buf, size_t size);
+ngx_chain_t * ngx_uring_writev_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit);
+ngx_chain_t *ngx_uring_sendfile_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit);
+static ssize_t ngx_uring_writev(ngx_connection_t *c, int nelts, int start_el);
+#if (NGX_USE_URING_SPLICE)
+static ssize_t ngx_uring_splice_sendfile(ngx_connection_t *c, ngx_buf_t *file, size_t size);
+#else
+static ssize_t ngx_uring_read_sendfile(ngx_connection_t *c, ngx_buf_t *file, size_t size);
+#endif
+
+
+
+static struct io_uring ring;
+static size_t linux_sendfile_bound;
+
+static ngx_str_t uring_name = ngx_string("io_uring");
+
+static ngx_command_t ngx_uring_commands[] = {
+
+ { ngx_string("uring_entries"),
+ NGX_EVENT_CONF|NGX_CONF_TAKE1,
+ ngx_conf_set_num_slot,
+ 0,
+ offsetof(ngx_uring_conf_t, entries),
+ NULL },
+
+ { ngx_string("linux_sendfile_bound"),
+ NGX_EVENT_CONF|NGX_CONF_TAKE1,
+ ngx_conf_set_num_slot,
+ 0,
+ offsetof(ngx_uring_conf_t, sendfile_bound),
+ NULL },
+
+ ngx_null_command
+};
+
+
+static ngx_event_module_t ngx_uring_module_ctx = {
+ &uring_name,
+ ngx_uring_create_conf, /* create configuration */
+ ngx_uring_init_conf, /* init configuration */
+
+ {
+ ngx_uring_add_event, /* add an event */
+ NULL, /* delete an event */
+ ngx_uring_add_event, /* enable an event */
+ NULL, /* disable an event */
+ ngx_uring_add_connection, /* add an connection */
+ ngx_uring_del_connection, /* delete an connection */
+ NULL, /* trigger a notify */
+ ngx_uring_process_events, /* process the events */
+ ngx_uring_init, /* init the events */
+ ngx_uring_done, /* done the events */
+ }
+};
+
+ngx_module_t ngx_uring_module = {
+ NGX_MODULE_V1,
+ &ngx_uring_module_ctx, /* module context */
+ ngx_uring_commands, /* module directives */
+ NGX_EVENT_MODULE, /* module type */
+ NULL, /* init master */
+ NULL, /* init module */
+ NULL, /* init process */
+ NULL, /* init thread */
+ NULL, /* exit thread */
+ NULL, /* exit process */
+ NULL, /* exit master */
+ NGX_MODULE_V1_PADDING
+};
+
+ngx_os_io_t ngx_uring_io = {
+ ngx_uring_recv,
+ ngx_uring_readv_chain,
+ NULL, /* udp recv */
+ ngx_uring_send,
+ NULL, /* udp send */
+ NULL, /* udp sendmsg chain */
+#if (1)
+ ngx_uring_sendfile_chain,
+ NGX_IO_SENDFILE
+#else
+ ngx_uring_writev_chain,
+ 0
+#endif
+};
+
+
+static ngx_int_t
+ngx_uring_init(ngx_cycle_t *cycle, ngx_msec_t timer)
+{
+ ngx_uring_conf_t *urcf;
+ struct io_uring_params params;
+
+ urcf = ngx_event_get_conf(cycle->conf_ctx, ngx_uring_module);
+
+ if(ring.ring_fd == 0){
+ ngx_memzero(&params, sizeof(params));
+ if (io_uring_queue_init_params(urcf->entries, &ring, &params) < 0) {
+ ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno,
+ "io_uring_queue_init_params() failed");
+ return NGX_ERROR;
+ }
+
+ if(!(params.features & IORING_FEAT_FAST_POLL)){
+ ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno,
+ "IORING_FEAT_FAST_POLL is not available");
+ return NGX_ERROR;
+ }
+ }
+
+ linux_sendfile_bound = urcf->sendfile_bound;
+
+ ngx_io = ngx_uring_io;
+
+ ngx_event_actions = ngx_uring_module_ctx.actions;
+
+#if (NGX_HAVE_CLEAR_EVENT)
+ ngx_event_flags = NGX_USE_CLEAR_EVENT
+#else
+ ngx_event_flags = NGX_USE_LEVEL_EVENT
+#endif
+ |NGX_USE_GREEDY_EVENT
+ |NGX_USE_URING_EVENT;
+
+ return NGX_OK;
+}
+
+
+
+static void
+ngx_uring_done(ngx_cycle_t *cycle)
+{
+ io_uring_queue_exit(&ring);
+ ngx_memset(&ring, 0, sizeof(ring));
+}
+
+
+static ngx_int_t
+ngx_uring_add_event(ngx_event_t *ev, ngx_int_t event, ngx_uint_t flags)
+{
+ ngx_connection_t *c;
+
+ c = ev->data;
+
+ if(event == NGX_READ_EVENT && c->read->accept){
+ if(ngx_uring_accept(c) == NGX_ERROR)
+ return NGX_ERROR;
+ }
+
+ ev->active = 1;
+ ev->ready = 1;
+
+ return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_uring_add_connection(ngx_connection_t *c)
+{
+ if(c->read->accept){
+ if(ngx_uring_accept(c) == NGX_ERROR)
+ return NGX_ERROR;
+ }
+
+ c->read->active = 1;
+ c->write->active = 1;
+ c->read->ready = 1;
+ c->write->ready = 1;
+
+ return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_uring_del_connection(ngx_connection_t *c, ngx_uint_t flags)
+{
+ c->read->active = 0;
+ c->write->active = 0;
+
+ return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_uring_process_events(ngx_cycle_t *cycle, ngx_msec_t timer, ngx_uint_t flags)
+{
+ unsigned head, count;
+ ngx_event_t *rev, *wev;
+ ngx_connection_t *c;
+ ngx_uring_info_t *ui;
+ struct io_uring_cqe *cqe;
+ struct io_uring_sqe *sqe;
+ struct __kernel_timespec ts;
+
+
+ /* NGX_TIMER_INFINITE == INFTIM */
+
+ ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0,
+ "io_uring timer: %M", timer);
+
+ if(timer != NGX_TIMER_INFINITE) {
+ if(timer >= 1000){
+ ts.tv_sec = timer / 1000;
+ ts.tv_nsec = 0;
+ } else{
+ ts.tv_sec = 0;
+ ts.tv_nsec = timer * 1000000;
+ }
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL) {
+ ngx_log_error(NGX_LOG_ALERT, cycle->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+
+ io_uring_prep_timeout(sqe, &ts, 1, 0);
+ io_uring_sqe_set_data(sqe, (void*)NGX_URING_TIMEOUT);
+ }
+
+ io_uring_submit_and_wait(&ring, 1);
+
+ if (flags & NGX_UPDATE_TIME || ngx_event_timer_alarm) {
+ ngx_time_update();
+ }
+
+ count = 0;
+
+ io_uring_for_each_cqe(&ring, head, cqe) {
+ ++count;
+
+ if(cqe->user_data == NGX_URING_TIMEOUT) {
+ if(count > 1) continue;
+
+ io_uring_cq_advance(&ring, count);
+
+ if(timer != NGX_TIMER_INFINITE){
+ return NGX_OK;
+ }
+
+ ngx_log_error(NGX_LOG_ALERT, cycle->log, 0,
+ "invalid timeout event");
+ return NGX_ERROR;
+ }
+
+ ui = (ngx_uring_info_t*)cqe->user_data;
+
+ c = ui->conn;
+
+ if (c->fd == -1) {
+ /*
+ * the stale event from a file descriptor
+ * that was just closed in this iteration
+ */
+ ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0,
+ "io_uring: stale event %p", c);
+ continue;
+ }
+
+ ngx_log_debug3(NGX_LOG_DEBUG_EVENT, cycle->log, 0,
+ "io_uring: fd:%d rq:%d d:%p",
+ c->fd, ui->ev, cqe->user_data);
+
+ rev = c->read;
+ wev = c->write;
+
+ switch (ui->ev)
+ {
+ case NGX_URING_ACCEPT:{
+ ngx_pfree(c->pool, ui);
+ if(cqe->res != -11) {
+ rev->uring_res = cqe->res;
+
+ rev->ready = 1;
+ rev->complete = 1;
+ rev->available = -1;
+
+ if (flags & NGX_POST_EVENTS) {
+ ngx_post_event(rev, &ngx_posted_accept_events);
+
+ } else {
+ rev->handler(rev);
+ }
+ }
+
+ rev->ready = 0;
+ rev->complete = 0;
+ rev->available = 1;
+
+ if(ngx_uring_accept(c) == NGX_ERROR)
+ return NGX_ERROR;
+ break;
+ }
+ case NGX_URING_READ:
+ case NGX_URING_READV:{
+ ngx_pfree(c->pool, ui);
+ rev->uring_pending -= 1;
+ rev->uring_res += cqe->res;
+
+ if(rev->uring_pending == 0){
+ rev->complete = 1;
+ rev->ready = 1;
+ rev->available = -1;
+
+ if (flags & NGX_POST_EVENTS) {
+ ngx_post_event(rev, &ngx_posted_events);
+
+ } else {
+ rev->handler(rev);
+ }
+ }
+ break;
+ }
+ case NGX_URING_SEND:
+ case NGX_URING_WRITEV:
+ case NGX_URING_SPLICE_FROM_PIPE:{
+ if(ui->buf) ngx_pfree(c->pool, ui->buf);
+ ngx_pfree(c->pool, ui);
+ wev->uring_pending -= 1;
+ wev->uring_res += cqe->res;
+
+ if(wev->uring_pending == 0) {
+ wev->complete = 1;
+ wev->ready = 1;
+ wev->available = -1;
+
+ if (flags & NGX_POST_EVENTS) {
+ ngx_post_event(wev, &ngx_posted_events);
+
+ } else {
+ wev->handler(wev);
+ }
+ }
+ break;
+ }
+ case NGX_URING_READFILE:
+ case NGX_URING_SPLICE_TO_PIPE:{
+ ngx_pfree(c->pool, ui);
+ break;
+ }
+ default:
+ ngx_log_error(NGX_LOG_ALERT, cycle->log, 0,
+ "io_uring invalid event type");
+ return NGX_ERROR;
+ }
+
+ }
+
+ io_uring_cq_advance(&ring, count);
+
+ if (count == 0) {
+ ngx_log_error(NGX_LOG_ALERT, cycle->log, 0,
+ "io_uring_submit_and_wait() returned no events without timeout");
+ return NGX_ERROR;
+ }
+
+ return NGX_OK;
+}
+
+
+static void *
+ngx_uring_create_conf(ngx_cycle_t *cycle)
+{
+ ngx_uring_conf_t *urcf;
+
+ urcf = ngx_palloc(cycle->pool, sizeof(ngx_uring_conf_t));
+ if (urcf == NULL) {
+ return NULL;
+ }
+
+ urcf->entries = NGX_CONF_UNSET;
+ urcf->sendfile_bound = NGX_CONF_UNSET;
+
+ return urcf;
+}
+
+
+static char *
+ngx_uring_init_conf(ngx_cycle_t *cycle, void *conf)
+{
+ ngx_uring_conf_t *urcf = conf;
+
+ ngx_conf_init_uint_value(urcf->entries, 32768);
+ ngx_conf_init_uint_value(urcf->sendfile_bound, 41984);
+
+ return NGX_CONF_OK;
+}
+
+
+ngx_int_t
+ngx_uring_accept(ngx_connection_t *c)
+{
+ struct io_uring_sqe *sqe;
+ ngx_uring_info_t *ui;
+
+ if(c->pool == NULL){
+ c->pool = ngx_create_pool(c->listening->pool_size, c->log);
+ if (c->pool == NULL) {
+ return NGX_ERROR;
+ }
+ }
+
+ if(c->sockaddr == NULL){
+ c->sockaddr = ngx_palloc(c->pool, sizeof(ngx_sockaddr_t));
+ if (c->sockaddr == NULL) {
+ return NGX_ERROR;
+ }
+ c->socklen = sizeof(ngx_sockaddr_t);
+ }
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_ACCEPT;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+
+ io_uring_prep_accept(sqe, c->fd, c->sockaddr, &c->socklen, 0);
+ io_uring_sqe_set_data(sqe, ui);
+
+ return NGX_OK;
+}
+
+
+ssize_t
+ngx_uring_recv(ngx_connection_t *c, u_char *buf, size_t size)
+{
+ ssize_t n;
+ ngx_event_t *rev;
+ ngx_uring_info_t *ui;
+ struct io_uring_sqe *sqe;
+
+ rev = c->read;
+
+ if(!rev->complete && rev->uring_pending) {
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0, "second uring_recv post");
+ return NGX_AGAIN;
+ }
+
+ if(rev->complete) {
+ n = rev->uring_res;
+ rev->uring_res = 0;
+ rev->available = 0;
+ rev->uring_pending = 0;
+ rev->complete = 0;
+
+ if(n == 0){
+ rev->ready = 0;
+ rev->eof = 1;
+ return 0;
+ }
+ if(n < 0){
+ ngx_connection_error(c, 0, "uring_recv() failed");
+ rev->error = 1;
+ return NGX_ERROR;
+ }
+
+ ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,
+ "uring_recv: fd:%d %ul of %z",
+ c->fd, n, size);
+
+ return n;
+ }
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_READ;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+
+ io_uring_prep_recv(sqe, c->fd, buf, size, 0);
+ io_uring_sqe_set_data(sqe, ui);
+
+ rev->complete = 0;
+ rev->ready = 0;
+ rev->uring_pending = 1;
+
+ return NGX_AGAIN;
+}
+
+ssize_t
+ngx_uring_readv_chain(ngx_connection_t *c, ngx_chain_t *chain, off_t limit)
+{
+ u_char *prev;
+ ssize_t n, size;
+ ngx_array_t vec;
+ ngx_event_t *rev;
+ struct iovec *iov;
+ ngx_uring_info_t *ui;
+ struct io_uring_sqe *sqe;
+
+ rev = c->read;
+
+ if(!rev->complete && rev->uring_pending) {
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0, "second uring_readv_chain post");
+ return NGX_AGAIN;
+ }
+
+ if(rev->complete) {
+ n = rev->uring_res;
+ rev->uring_res = 0;
+ rev->available = 0;
+ rev->uring_pending = 0;
+ rev->complete = 0;
+
+ if(n == 0){
+ rev->ready = 0;
+ rev->eof = 1;
+ return 0;
+ }
+ if(n < 0){
+ ngx_connection_error(c, 0, "uring_readv() failed");
+ rev->error = 1;
+ return NGX_ERROR;
+ }
+
+ ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,
+ "uring_readv: fd:%d %ul of %z",
+ c->fd, n, size);
+
+ return n;
+ }
+
+ prev = NULL;
+ iov = NULL;
+ size = 0;
+
+ vec.elts = rev->uring_iov;
+ vec.nelts = 0;
+ vec.size = sizeof(struct iovec);
+ vec.nalloc = NGX_IOVS_PREALLOCATE;
+ vec.pool = c->pool;
+
+ /* coalesce the neighbouring bufs */
+
+ while (chain) {
+ n = chain->buf->end - chain->buf->last;
+
+ if (limit) {
+ if (size >= limit) {
+ break;
+ }
+
+ if (size + n > limit) {
+ n = (ssize_t) (limit - size);
+ }
+ }
+
+ if (prev == chain->buf->last) {
+ iov->iov_len += n;
+
+ } else {
+ if (vec.nelts >= IOV_MAX) {
+ break;
+ }
+
+ iov = ngx_array_push(&vec);
+ if (iov == NULL) {
+ return NGX_ERROR;
+ }
+
+ iov->iov_base = (void *) chain->buf->last;
+ iov->iov_len = n;
+ }
+
+ size += n;
+ prev = chain->buf->end;
+ chain = chain->next;
+ }
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
+ "readv: %ui, last:%uz", vec.nelts, iov->iov_len);
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_READV;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+
+ io_uring_prep_readv(sqe, c->fd, (struct iovec *) vec.elts, vec.nelts, 0);
+ io_uring_sqe_set_data(sqe, ui);
+
+ rev->complete = 0;
+ rev->ready = 0;
+ rev->uring_pending = 1;
+
+ return NGX_AGAIN;
+}
+
+#define NGX_SENDFILE_MAXSIZE 2147483647L
+
+ssize_t
+ngx_uring_send(ngx_connection_t *c, u_char *buf, size_t size)
+{
+ ssize_t n;
+ ngx_event_t *wev;
+ ngx_uring_info_t *ui;
+ struct io_uring_sqe *sqe;
+
+ wev = c->write;
+
+ if(wev->uring_pending && !wev->complete){
+ return NGX_AGAIN;
+ }
+
+ if(wev->complete){
+ n = wev->uring_res;
+ wev->complete = 0;
+ wev->uring_pending = 0;
+ wev->uring_res = 0;
+ wev->ready = 1;
+
+ ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,
+ "send: fd:%d %z of %uz", c->fd, n, size);
+
+ if (n == 0) {
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0, "send() returned zero");
+ wev->ready = 0;
+ return n;
+ }
+
+ if (n > 0) {
+ if (n < (ssize_t) size) {
+ wev->ready = 0;
+ }
+
+ c->sent += n;
+
+ return n;
+ }
+
+
+ wev->error = 1;
+ (void) ngx_connection_error(c, 0, "send() failed");
+ return NGX_ERROR;
+
+ }
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_SEND;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+
+ io_uring_prep_send(sqe, c->fd, buf, size, 0);
+ io_uring_sqe_set_data(sqe, ui);
+
+ wev->uring_rq_size = size;
+ wev->uring_pending = 1;
+ wev->complete = 0;
+ wev->ready = 0;
+
+ return NGX_AGAIN;
+}
+
+static ssize_t
+ngx_uring_writev(ngx_connection_t *c, int nelts, int start_el)
+{
+ ngx_uring_info_t *ui;
+ struct io_uring_sqe *sqe;
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_WRITEV;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+
+ io_uring_prep_writev(sqe, c->fd, &c->write->uring_iov[start_el], nelts - start_el , 0);
+ io_uring_sqe_set_data(sqe, ui);
+
+ return NGX_AGAIN;
+}
+
+
+ngx_chain_t *
+ngx_uring_writev_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit)
+{
+ u_char *prev;
+ ssize_t n, sent, size;
+ off_t send;
+ ngx_chain_t *cl;
+ ngx_event_t *wev;
+ int nelts, start_el;
+
+ wev = c->write;
+
+ if(wev->uring_pending && !wev->complete){
+ return in;
+ }
+
+ if(wev->complete){
+ sent = wev->uring_res;
+
+ wev->complete = 0;
+ wev->uring_pending = 0;
+ wev->uring_res = 0;
+ wev->ready = 1;
+
+ if(sent != wev->uring_rq_size){
+ ngx_connection_error(c, 0, "uring_writev_chain failed");
+ return NGX_CHAIN_ERROR;
+ }
+
+ c->sent += sent;
+ in = ngx_chain_update_sent(in, sent);
+
+ return in;
+ }
+
+ send = 0;
+ nelts = 0;
+ prev = NULL;
+ start_el = 0;
+
+
+ /* the maximum limit size is the maximum size_t value - the page size */
+
+ if (limit == 0 || limit > (off_t) (NGX_MAX_SIZE_T_VALUE - ngx_pagesize)) {
+ limit = NGX_MAX_SIZE_T_VALUE - ngx_pagesize;
+ }
+
+ for (cl = in;
+ cl && nelts < NGX_IOVS_PREALLOCATE && send < limit;
+ )
+ {
+
+ if (ngx_buf_special(cl->buf)) {
+ cl = cl->next;
+ continue;
+ }
+
+ if (cl && cl->buf->in_file) {
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "file buf in writev "
+ "t:%d r:%d f:%d %p %p-%p %p %O-%O",
+ cl->buf->temporary,
+ cl->buf->recycled,
+ cl->buf->in_file,
+ cl->buf->start,
+ cl->buf->pos,
+ cl->buf->last,
+ cl->buf->file,
+ cl->buf->file_pos,
+ cl->buf->file_last);
+ ngx_debug_point();
+
+ return NGX_CHAIN_ERROR;
+ }
+
+
+ size = cl->buf->last - cl->buf->pos;
+ if (send + size > limit) {
+ size = (u_long) (limit - send);
+ }
+
+ if (prev == cl->buf->pos) {
+ wev->uring_iov[nelts - 1].iov_len += cl->buf->last - cl->buf->pos;
+
+ } else {
+ ++nelts;
+ if (nelts >= NGX_IOVS_PREALLOCATE) {
+ wev->error = 1;
+ return NGX_CHAIN_ERROR;
+ }
+ wev->uring_iov[nelts - 1].iov_base = (void *) cl->buf->pos;
+ wev->uring_iov[nelts - 1].iov_len = cl->buf->last - cl->buf->pos;
+ }
+ prev = cl->buf->last;
+ send += size;
+ cl = cl->next;
+ }
+
+ if(nelts - start_el > 0){
+ n = ngx_uring_writev(c, nelts, start_el);
+
+ if (n == NGX_ERROR) {
+ return NGX_CHAIN_ERROR;
+ }
+
+ sent = 0;
+ }
+
+ wev->uring_rq_size = send;
+ wev->uring_pending = 1;
+ wev->complete = 0;
+ wev->ready = 0;
+
+ return in;
+}
+
+#if (NGX_USE_URING_SPLICE)
+static ssize_t
+ngx_uring_splice_sendfile(ngx_connection_t *c, ngx_buf_t *file, size_t size)
+{
+ ngx_uring_info_t *ui;
+ struct io_uring_sqe *sqe;
+#if (NGX_HAVE_SENDFILE64)
+ off_t offset;
+#else
+ int32_t offset;
+#endif
+#if (NGX_HAVE_SENDFILE64)
+ offset = file->file_pos;
+#else
+ offset = (int32_t) file->file_pos;
+#endif
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_SPLICE_TO_PIPE;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+ io_uring_prep_splice(sqe, file->file->fd, offset,
+ c->write->uring_splice_pipe[1], -1, size, SPLICE_F_MOVE | SPLICE_F_MORE);
+ sqe->flags = IOSQE_IO_LINK;
+ io_uring_sqe_set_data(sqe, ui);
+
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_SPLICE_FROM_PIPE;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+ io_uring_prep_splice(sqe, c->write->uring_splice_pipe[0], -1,
+ c->fd, -1, size, SPLICE_F_MOVE | SPLICE_F_MORE);
+ io_uring_sqe_set_data(sqe, ui);
+
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
+ "uring_splice_sendfile: @%O %uz", file->file_pos, size);
+
+ return NGX_AGAIN;
+}
+#else
+static ssize_t
+ngx_uring_read_sendfile(ngx_connection_t *c, ngx_buf_t *file, size_t size)
+{
+ ngx_uring_info_t *ui;
+ struct io_uring_sqe *sqe;
+ void *buf;
+#if (NGX_HAVE_SENDFILE64)
+ off_t offset;
+#else
+ int32_t offset;
+#endif
+#if (NGX_HAVE_SENDFILE64)
+ offset = file->file_pos;
+#else
+ offset = (int32_t) file->file_pos;
+#endif
+
+ buf = ngx_palloc(c->pool, size);
+ if(buf == NULL){
+ return NGX_ERROR;
+ }
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_READFILE;
+ ui->buf = NULL;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+
+ io_uring_prep_read(sqe, file->file->fd, buf, size, offset );
+ sqe->flags = IOSQE_IO_LINK;
+ io_uring_sqe_set_data(sqe, ui);
+
+
+ ui = ngx_palloc(c->pool, sizeof(ngx_uring_info_t));
+ if(ui == NULL){
+ return NGX_ERROR;
+ }
+ ui->conn = c;
+ ui->ev = NGX_URING_SEND;
+ ui->buf = buf;
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+ "io_uring prep event: fd:%d op:%d ",
+ c->fd, ui->ev);
+
+ sqe = io_uring_get_sqe(&ring);
+ if(sqe == NULL){
+ ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+ "io_uring_get_sqe() failed");
+ return NGX_ERROR;
+ }
+ io_uring_prep_send(sqe, c->fd, buf, size, 0);
+ io_uring_sqe_set_data(sqe, ui);
+
+
+ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
+ "uring_splice_sendfile: @%O %uz", file->file_pos, size);
+
+ return NGX_AGAIN;
+}
+#endif
+
+
+ngx_chain_t *
+ngx_uring_sendfile_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit)
+{
+ u_char *prev;
+ off_t send;
+ size_t file_size;
+ ngx_buf_t *file;
+ ngx_event_t *wev;
+ ngx_chain_t *cl;
+ ssize_t n, sent, size;
+ int nelts, start_el, pending;
+
+ wev = c->write;
+
+ if(wev->uring_pending && !wev->complete){
+ return in;
+ }
+
+ if(wev->complete){
+ sent = wev->uring_res;
+
+ wev->complete = 0;
+ wev->uring_pending = 0;
+ wev->uring_res = 0;
+ wev->ready = 1;
+
+ if(sent != wev->uring_rq_size){
+ ngx_connection_error(c, 0, "uring_writev_chain failed");
+ return NGX_CHAIN_ERROR;
+ }
+
+ c->sent += sent;
+ in = ngx_chain_update_sent(in, sent);
+
+ return in;
+ }
+ sent = 0;
+ send = 0;
+ nelts = 0;
+ prev = NULL;
+ start_el = 0;
+ pending = 0;
+
+ if (limit == 0 || limit > (off_t) (NGX_SENDFILE_MAXSIZE - ngx_pagesize)) {
+ limit = NGX_SENDFILE_MAXSIZE - ngx_pagesize;
+ }
+
+ for (cl = in;
+ cl && nelts < NGX_IOVS_PREALLOCATE && send < limit;
+ )
+ {
+ if (ngx_buf_special(cl->buf)) {
+ cl = cl->next;
+ continue;
+ }
+
+ if (cl->buf->in_file) {
+ if(nelts > 0){
+ ++pending;
+ n = ngx_uring_writev(c, nelts, start_el);
+
+ if (n == NGX_ERROR) {
+ return NGX_CHAIN_ERROR;
+ }
+
+ start_el += nelts;
+ prev = NULL;
+ }
+
+ file = cl->buf;
+
+ /* coalesce the neighbouring file bufs */
+
+ file_size = (size_t) ngx_chain_coalesce_file(&cl, limit - send);
+
+ send += file_size;
+
+ if (file_size == 0) {
+ ngx_debug_point();
+ return NGX_CHAIN_ERROR;
+ }
+
+ if(file_size >= linux_sendfile_bound) {
+ io_uring_submit(&ring);
+
+ n = ngx_linux_sendfile(c, file, file_size);
+
+ if (n == NGX_ERROR) {
+ return NGX_CHAIN_ERROR;
+ }
+
+ if(n != NGX_AGAIN){
+ sent += n;
+ continue;
+ }
+ }
+
+#if (NGX_USE_URING_SPLICE)
+ n = ngx_uring_splice_sendfile(c, file, file_size);
+#else
+ n = ngx_uring_read_sendfile(c, file, file_size);
+#endif
+
+ if (n == NGX_ERROR) {
+ wev->error = 1;
+ return NGX_CHAIN_ERROR;
+ }
+
+ ++pending;
+ continue;
+ }
+
+ size = cl->buf->last - cl->buf->pos;
+ if (send + size > limit) {
+ size = (u_long) (limit - send);
+ }
+
+ if (prev == cl->buf->pos) {
+ wev->uring_iov[nelts - 1].iov_len += cl->buf->last - cl->buf->pos;
+
+ } else {
+ ++nelts;
+ if (nelts >= NGX_IOVS_PREALLOCATE) {
+ wev->error = 1;
+ return NGX_CHAIN_ERROR;
+ }
+ wev->uring_iov[nelts - 1].iov_base = (void *) cl->buf->pos;
+ wev->uring_iov[nelts - 1].iov_len = cl->buf->last - cl->buf->pos;
+ }
+
+ prev = cl->buf->last;
+ send += size;
+ cl = cl->next;
+ }
+
+ if(nelts - start_el > 0){
+ ++pending;
+ n = ngx_uring_writev(c, nelts, start_el);
+
+ if (n == NGX_ERROR) {
+ return NGX_CHAIN_ERROR;
+ }
+ }
+
+ wev->uring_rq_size = send;
+ wev->uring_pending = pending;
+ wev->complete = 0;
+ wev->ready = 0;
+ wev->uring_res = sent;
+
+ return in;
+}
diff -r ce34097321cf -r d7f0f0b78a24 src/event/ngx_event.c
--- a/src/event/ngx_event.c Mon Nov 23 10:49:15 2020 +0000
+++ b/src/event/ngx_event.c Mon Nov 23 10:52:02 2020 +0000
@@ -17,6 +17,7 @@
extern ngx_module_t ngx_eventport_module;
extern ngx_module_t ngx_devpoll_module;
extern ngx_module_t ngx_epoll_module;
+extern ngx_module_t ngx_uring_module;
extern ngx_module_t ngx_select_module;


_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Tue, 24 Nov 2020 03:46:07 -0500
http://www.ldmicj.icu/read.php?29,290034,290034#msg-290034 [PATCH 7 of 8] Added creation and close of pipes for sendfile of ngx_uring_module (no replies) http://www.ldmicj.icu/read.php?29,290034,290034#msg-290034 # User SoYun Seong <thdbsdox12@gmail.com>
# Date 1606129296 0
# Mon Nov 23 11:01:36 2020 +0000
# Node ID 09dfe4a92414513c6bd3c18d871e8a76ed19c3d7
# Parent 57e46378828172e06b32b11d4c059ff0a7213d0d
Added creation and close of pipes for sendfile of ngx_uring_module.

Creates pipes for asynchronous sendfile of ngx_uring module in ngx_get_connection() and destroy them in ngx_free_connection(). However this code is disabled for the performance issue.

diff -r 57e463788281 -r 09dfe4a92414 src/core/ngx_connection.c
--- a/src/core/ngx_connection.c Mon Nov 23 10:59:09 2020 +0000
+++ b/src/core/ngx_connection.c Mon Nov 23 11:01:36 2020 +0000
@@ -1152,6 +1152,14 @@

wev->write = 1;

+#if (NGX_USE_URING_SPLICE)
+ if(ngx_event_flags & NGX_USE_URING_EVENT) {
+ if(pipe(c->write->uring_splice_pipe) < 0){
+ return NULL;
+ }
+ }
+#endif
+
return c;
}

@@ -1166,6 +1174,13 @@
if (ngx_cycle->files && ngx_cycle->files[c->fd] == c) {
ngx_cycle->files[c->fd] = NULL;
}
+
+#if (NGX_USE_URING_SPLICE)
+ if(ngx_event_flags & NGX_USE_URING_EVENT) {
+ close(c->write->uring_splice_pipe[0]);
+ close(c->write->uring_splice_pipe[1]);
+ }
+#endif
}


_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Tue, 24 Nov 2020 03:46:07 -0500
http://www.ldmicj.icu/read.php?29,290033,290033#msg-290033 [PATCH 6 of 8] Disabled freeing recv buffer when asynchronous recv IO is in running (no replies) http://www.ldmicj.icu/read.php?29,290033,290033#msg-290033 # User SoYun Seong <thdbsdox12@gmail.com>
# Date 1606129149 0
# Mon Nov 23 10:59:09 2020 +0000
# Node ID 57e46378828172e06b32b11d4c059ff0a7213d0d
# Parent bdf6d23007d53d9b76a7666c8db7ece864e31038
Disabled freeing recv buffer when asynchronous recv IO is in running.

diff -r bdf6d23007d5 -r 57e463788281 src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c Mon Nov 23 10:56:29 2020 +0000
+++ b/src/http/ngx_http_request.c Mon Nov 23 10:59:09 2020 +0000
@@ -444,8 +444,10 @@
* We are trying to not hold c->buffer's memory for an idle connection.
*/

- if (ngx_pfree(c->pool, b->start) == NGX_OK) {
- b->start = NULL;
+ if(!(ngx_event_flags & NGX_USE_URING_EVENT)) {
+ if (ngx_pfree(c->pool, b->start) == NGX_OK) {
+ b->start = NULL;
+ }
}

return;
@@ -3317,13 +3319,15 @@
* c->buffer's memory for a keepalive connection.
*/

- if (ngx_pfree(c->pool, b->start) == NGX_OK) {
-
- /*
- * the special note that c->buffer's memory was freed
- */
-
- b->pos = NULL;
+ if(!(ngx_event_flags & NGX_USE_URING_EVENT)) {
+ if (ngx_pfree(c->pool, b->start) == NGX_OK) {
+
+ /*
+ * the special note that c->buffer's memory was freed
+ */
+
+ b->pos = NULL;
+ }
}

return;
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Tue, 24 Nov 2020 03:46:07 -0500
http://www.ldmicj.icu/read.php?29,290032,290032#msg-290032 [PATCH 5 of 8] Added to process asynchronous accept (no replies) http://www.ldmicj.icu/read.php?29,290032,290032#msg-290032 # User SoYun Seong <thdbsdox12@gmail.com>
# Date 1606128989 0
# Mon Nov 23 10:56:29 2020 +0000
# Node ID bdf6d23007d53d9b76a7666c8db7ece864e31038
# Parent d7f0f0b78a24724c362cd0306dc6bbed82c66e62
Added to process asynchronous accept

use a result of a requested accept to ngx_uring_module, instead of calling accept() or accept4() system call.

diff -r d7f0f0b78a24 -r bdf6d23007d5 src/event/ngx_event_accept.c
--- a/src/event/ngx_event_accept.c Mon Nov 23 10:52:02 2020 +0000
+++ b/src/event/ngx_event_accept.c Mon Nov 23 10:56:29 2020 +0000
@@ -55,15 +55,22 @@
do {
socklen = sizeof(ngx_sockaddr_t);

+ if(ngx_event_flags & NGX_USE_URING_EVENT)
+ {
+ s = ev->uring_res;
+ sa.sockaddr = *(lc->sockaddr);
+ socklen = lc->socklen;
+ } else {
#if (NGX_HAVE_ACCEPT4)
- if (use_accept4) {
- s = accept4(lc->fd, &sa.sockaddr, &socklen, SOCK_NONBLOCK);
- } else {
+ if (use_accept4) {
+ s = accept4(lc->fd, &sa.sockaddr, &socklen, SOCK_NONBLOCK);
+ } else {
+ s = accept(lc->fd, &sa.sockaddr, &socklen);
+ }
+#else
s = accept(lc->fd, &sa.sockaddr, &socklen);
+#endif
}
-#else
- s = accept(lc->fd, &sa.sockaddr, &socklen);
-#endif

if (s == (ngx_socket_t) -1) {
err = ngx_socket_errno;
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Tue, 24 Nov 2020 03:46:07 -0500
http://www.ldmicj.icu/read.php?29,290031,290031#msg-290031 [PATCH 3 of 8] Make ngx_linux_sendfile() to global function (no replies) http://www.ldmicj.icu/read.php?29,290031,290031#msg-290031 # User SoYun Seong <thdbsdox12@gmail.com>
# Date 1606128555 0
# Mon Nov 23 10:49:15 2020 +0000
# Node ID ce34097321cf3cb7f9de55075bd1788577bb1ad8
# Parent 3bbe4905410b04bdaad4c956f6fdb7d0612ddf5b
Make ngx_linux_sendfile() to global function

diff -r 3bbe4905410b -r ce34097321cf src/os/unix/ngx_linux.h
--- a/src/os/unix/ngx_linux.h Mon Nov 23 10:47:13 2020 +0000
+++ b/src/os/unix/ngx_linux.h Mon Nov 23 10:49:15 2020 +0000
@@ -12,5 +12,8 @@
ngx_chain_t *ngx_linux_sendfile_chain(ngx_connection_t *c, ngx_chain_t *in,
off_t limit);

+ssize_t ngx_linux_sendfile(ngx_connection_t *c, ngx_buf_t *file,
+ size_t size);
+

#endif /* _NGX_LINUX_H_INCLUDED_ */
diff -r 3bbe4905410b -r ce34097321cf src/os/unix/ngx_linux_sendfile_chain.c
--- a/src/os/unix/ngx_linux_sendfile_chain.c Mon Nov 23 10:47:13 2020 +0000
+++ b/src/os/unix/ngx_linux_sendfile_chain.c Mon Nov 23 10:49:15 2020 +0000
@@ -10,9 +10,6 @@
#include <ngx_event.h>


-static ssize_t ngx_linux_sendfile(ngx_connection_t *c, ngx_buf_t *file,
- size_t size);
-
#if (NGX_THREADS)
#include <ngx_thread_pool.h>

@@ -226,7 +223,7 @@
}


-static ssize_t
+ssize_t
ngx_linux_sendfile(ngx_connection_t *c, ngx_buf_t *file, size_t size)
{
#if (NGX_HAVE_SENDFILE64)
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Tue, 24 Nov 2020 03:46:07 -0500
http://www.ldmicj.icu/read.php?29,290030,290030#msg-290030 [PATCH 2 of 8] Added variables and defined a new flag for io_uring module (no replies) http://www.ldmicj.icu/read.php?29,290030,290030#msg-290030 # User SoYun Seong <thdbsdox12@gmail.com>
# Date 1606128433 0
# Mon Nov 23 10:47:13 2020 +0000
# Node ID 3bbe4905410b04bdaad4c956f6fdb7d0612ddf5b
# Parent 1cb2b354e262e10a8e8606e238ca0fa279f70709
Added variables and defined a new flag for io_uring module.

For io_uring event module, added member variables to ngx_event_s structure. ‘uring_res?is used to pass the result of IO to a read or write event. ‘uring_pending?means the total requested IO count of the event and ‘uring_rq_size?means the total IO size of the event. Both variables is used to check completion of the request. ‘uring_iov?is used to do asynchronous vector IO. ‘uring_splice_pipe?is used to process asynchronous sendfile. Also added an io_uring event flag and defined requested IO type for io_uring.

diff -r 1cb2b354e262 -r 3bbe4905410b src/event/ngx_event.h
--- a/src/event/ngx_event.h Mon Nov 23 10:08:50 2020 +0000
+++ b/src/event/ngx_event.h Mon Nov 23 10:47:13 2020 +0000
@@ -107,6 +107,14 @@
ngx_event_ovlp_t ovlp;
#endif

+#if (NGX_HAVE_URING)
+ int uring_res;
+ ngx_uint_t uring_pending;
+ ssize_t uring_rq_size;
+ struct iovec uring_iov[NGX_IOVS_PREALLOCATE];
+ int uring_splice_pipe[2];
+#endif
+
ngx_uint_t index;

ngx_log_t *log;
@@ -271,6 +279,11 @@
*/
#define NGX_USE_VNODE_EVENT 0x00002000

+/*
+ * The event filter is io_uring.
+ */
+#define NGX_USE_URING_EVENT 0x00004000
+

/*
* The event filter is deleted just before the closing file.
@@ -390,6 +403,19 @@
#define NGX_IOCP_CONNECT 2
#endif

+#if (NGX_HAVE_URING)
+#define NGX_URING_ACCEPT 0
+#define NGX_URING_READ 1
+#define NGX_URING_READV 2
+#define NGX_URING_SEND 3
+#define NGX_URING_WRITEV 4
+#define NGX_URING_SPLICE_TO_PIPE 5
+#define NGX_URING_SPLICE_FROM_PIPE 6
+#define NGX_URING_READFILE 7
+#define NGX_URING_TIMEOUT 1
+#define NGX_USE_URING_SPLICE 0
+#endif
+

#if (NGX_TEST_BUILD_EPOLL)
#define NGX_EXCLUSIVE_EVENT 0
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
SoYun Seong Nginx Development Tue, 24 Nov 2020 03:46:07 -0500
http://www.ldmicj.icu/read.php?29,290021,290021#msg-290021 [njs] Ignoring pcre_study() error. (no replies) http://www.ldmicj.icu/read.php?29,290021,290021#msg-290021 branches:
changeset: 1569:e51da8c71f26
user: Valentin Bartenev <vbart@nginx.com>
date: Fri Nov 20 12:29:30 2020 +0300
description:
Ignoring pcre_study() error.

It provides optional optimization that shouldn't be fatal for regex compilation.

diffstat:

src/njs_pcre.c | 4 +---
1 files changed, 1 insertions(+), 3 deletions(-)

diffs (15 lines):

diff -r c947a300b96c -r e51da8c71f26 src/njs_pcre.c
--- a/src/njs_pcre.c Wed Nov 18 18:09:11 2020 +0000
+++ b/src/njs_pcre.c Fri Nov 20 12:29:30 2020 +0300
@@ -94,10 +94,8 @@ njs_regex_compile(njs_regex_t *regex, u_
regex->extra = pcre_study(regex->code, 0, &errstr);

if (njs_slow_path(errstr != NULL)) {
- njs_alert(ctx->trace, NJS_LEVEL_ERROR,
+ njs_alert(ctx->trace, NJS_LEVEL_WARN,
"pcre_study(\"%s\") failed: %s", pattern, errstr);
-
- goto done;
}

err = pcre_fullinfo(regex->code, NULL, PCRE_INFO_CAPTURECOUNT,
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Valentin Bartenev Nginx Development Fri, 20 Nov 2020 04:34:07 -0500
http://www.ldmicj.icu/read.php?29,290016,290016#msg-290016 Wrong list I know, but need www.ldmicj.icu mail admin contact (no replies) http://www.ldmicj.icu/read.php?29,290016,290016#msg-290016 the direct contact for someone who can fix www.ldmicj.icu mail delivery
issues?

Getting a number of Rejected Messages because of something that changed
on your end.


Thomas

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Thomas Ward Nginx Development Thu, 19 Nov 2020 14:26:03 -0500
http://www.ldmicj.icu/read.php?29,290014,290014#msg-290014 [nginx] gRPC: RST_STREAM(NO_ERROR) handling after "trailer only" responses. (no replies) http://www.ldmicj.icu/read.php?29,290014,290014#msg-290014 branches:
changeset: 7746:88eca63261c3
user: Pavel Pautov <p.pautov@f5.com>
date: Wed Nov 18 18:41:16 2020 -0800
description:
gRPC: RST_STREAM(NO_ERROR) handling after "trailer only" responses.

Similarly to the problem fixed in 2096b21fcd10 (ticket #1792),
when a "trailer only" gRPC response (that is, a response with the
END_STREAM flag in the HEADERS frame) was immediately followed by
RST_STREAM(NO_ERROR) in the data preread along with the response
header, RST_STREAM wasn't properly skipped and caused "upstream
rejected request with error 0" errors.
Observed with "unknown service" gRPC errors returned by grpc-go.

Fix is to set ctx->done if we are going to parse additional data,
so the RST_STREAM(NO_ERROR) is properly skipped. Additionally, now
ngx_http_grpc_filter() will complain about frames sent for closed
stream if there are any.

diffstat:

src/http/modules/ngx_http_grpc_module.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)

diffs (11 lines):

diff -r f57a478aa16d -r 88eca63261c3 src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c Thu Nov 19 17:15:22 2020 +0000
+++ b/src/http/modules/ngx_http_grpc_module.c Wed Nov 18 18:41:16 2020 -0800
@@ -1969,6 +1969,7 @@ ngx_http_grpc_filter_init(void *data)
}

u->length = 0;
+ ctx->done = 1;

} else {
u->length = 1;
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Pavel Pautov Nginx Development Thu, 19 Nov 2020 14:18:08 -0500
http://www.ldmicj.icu/read.php?29,290012,290012#msg-290012 [nginx] Use .Mt to mark up email addresses. (no replies) http://www.ldmicj.icu/read.php?29,290012,290012#msg-290012 branches:
changeset: 7745:f57a478aa16d
user: Sergey Kandaurov <pluknet@nginx.com>
date: Thu Nov 19 17:15:22 2020 +0000
description:
Use .Mt to mark up email addresses.

diffstat:

docs/man/nginx.8 | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)

diffs (16 lines):

diff -r f18db38a9826 -r f57a478aa16d docs/man/nginx.8
--- a/docs/man/nginx.8 Thu Nov 19 16:59:00 2020 +0000
+++ b/docs/man/nginx.8 Thu Nov 19 17:15:22 2020 +0000
@@ -205,10 +205,10 @@ Development of
started in 2002, with the first public release on October 4, 2004.
.Sh AUTHORS
.An -nosplit
-.An Igor Sysoev Aq igor@sysoev.ru .
+.An Igor Sysoev Aq Mt igor@sysoev.ru .
.Pp
This manual page was originally written by
-.An Sergey A. Osokin Aq osa@FreeBSD.org.ru
+.An Sergey A. Osokin Aq Mt osa@FreeBSD.org.ru
as a result of compiling many
.Nm
documents from all over the world.
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Sergey Kandaurov Nginx Development Thu, 19 Nov 2020 13:58:13 -0500
http://www.ldmicj.icu/read.php?29,290008,290008#msg-290008 [nginx] Core: "-e" command line option. (no replies) http://www.ldmicj.icu/read.php?29,290008,290008#msg-290008 branches:
changeset: 7744:f18db38a9826
user: Igor Ippolitov <iippolitov@nginx.com>
date: Thu Nov 19 16:59:00 2020 +0000
description:
Core: "-e" command line option.

When installing or running from a non-root user it is sometimes required to
override default, compiled in error log path. There was no way to do this
without rebuilding the binary (ticket #147).

This patch introduced "-e" command line option which allows one to override
compiled in error log path.

diffstat:

auto/configure | 4 ++++
docs/man/nginx.8 | 9 ++++++++-
src/core/nginx.c | 41 +++++++++++++++++++++++++++++++++++++----
src/core/ngx_cycle.c | 9 +++++++++
src/core/ngx_cycle.h | 1 +
src/core/ngx_log.c | 19 ++++++++-----------
src/core/ngx_log.h | 2 +-
7 files changed, 68 insertions(+), 17 deletions(-)

diffs (233 lines):

diff -r 4b1299b1856a -r f18db38a9826 auto/configure
--- a/auto/configure Tue Nov 10 17:13:20 2020 +0300
+++ b/auto/configure Thu Nov 19 16:59:00 2020 +0000
@@ -87,6 +87,10 @@ have=NGX_PID_PATH value="\"$NGX_PID_PATH
have=NGX_LOCK_PATH value="\"$NGX_LOCK_PATH\"" . auto/define
have=NGX_ERROR_LOG_PATH value="\"$NGX_ERROR_LOG_PATH\"" . auto/define

+if [ ".$NGX_ERROR_LOG_PATH" = "." ]; then
+ have=NGX_ERROR_LOG_STDERR . auto/have
+fi
+
have=NGX_HTTP_LOG_PATH value="\"$NGX_HTTP_LOG_PATH\"" . auto/define
have=NGX_HTTP_CLIENT_TEMP_PATH value="\"$NGX_HTTP_CLIENT_TEMP_PATH\""
. auto/define
diff -r 4b1299b1856a -r f18db38a9826 docs/man/nginx.8
--- a/docs/man/nginx.8 Tue Nov 10 17:13:20 2020 +0300
+++ b/docs/man/nginx.8 Thu Nov 19 16:59:00 2020 +0000
@@ -25,7 +25,7 @@
.\" SUCH DAMAGE.
.\"
.\"
-.Dd December 5, 2019
+.Dd November 5, 2020
.Dt NGINX 8
.Os
.Sh NAME
@@ -35,6 +35,7 @@
.Nm
.Op Fl ?hqTtVv
.Op Fl c Ar file
+.Op Fl e Ar file
.Op Fl g Ar directives
.Op Fl p Ar prefix
.Op Fl s Ar signal
@@ -54,6 +55,12 @@ Print help.
.It Fl c Ar file
Use an alternative configuration
.Ar file .
+.It Fl e Ar file
+Use an alternative error log
+.Ar file .
+Special value
+.Cm stderr
+indicates that the standard error output should be used.
.It Fl g Ar directives
Set global configuration directives.
See
diff -r 4b1299b1856a -r f18db38a9826 src/core/nginx.c
--- a/src/core/nginx.c Tue Nov 10 17:13:20 2020 +0300
+++ b/src/core/nginx.c Thu Nov 19 16:59:00 2020 +0000
@@ -183,6 +183,7 @@ static ngx_uint_t ngx_show_help;
static ngx_uint_t ngx_show_version;
static ngx_uint_t ngx_show_configure;
static u_char *ngx_prefix;
+static u_char *ngx_error_log;
static u_char *ngx_conf_file;
static u_char *ngx_conf_params;
static char *ngx_signal;
@@ -230,7 +231,7 @@ main(int argc, char *const *argv)
ngx_pid = ngx_getpid();
ngx_parent = ngx_getppid();

- log = ngx_log_init(ngx_prefix);
+ log = ngx_log_init(ngx_prefix, ngx_error_log);
if (log == NULL) {
return 1;
}
@@ -393,9 +394,9 @@ ngx_show_version_info(void)

if (ngx_show_help) {
ngx_write_stderr(
- "Usage: nginx [-?hvVtTq] [-s signal] [-c filename] "
- "[-p prefix] [-g directives]" NGX_LINEFEED
- NGX_LINEFEED
+ "Usage: nginx [-?hvVtTq] [-s signal] [-p prefix]" NGX_LINEFEED
+ " [-e filename] [-c filename] [-g directives]"
+ NGX_LINEFEED NGX_LINEFEED
"Options:" NGX_LINEFEED
" -?,-h : this help" NGX_LINEFEED
" -v : show version and exit" NGX_LINEFEED
@@ -414,6 +415,12 @@ ngx_show_version_info(void)
#else
" -p prefix : set prefix path (default: NONE)" NGX_LINEFEED
#endif
+ " -e filename : set error log file (default: "
+#ifdef NGX_ERROR_LOG_STDERR
+ "stderr)" NGX_LINEFEED
+#else
+ NGX_ERROR_LOG_PATH ")" NGX_LINEFEED
+#endif
" -c filename : set configuration file (default: " NGX_CONF_PATH
")" NGX_LINEFEED
" -g directives : set global directives out of configuration "
@@ -800,6 +807,24 @@ ngx_get_options(int argc, char *const *a
ngx_log_stderr(0, "option \"-p\" requires directory name");
return NGX_ERROR;

+ case 'e':
+ if (*p) {
+ ngx_error_log = p;
+
+ } else if (argv[++i]) {
+ ngx_error_log = (u_char *) argv[i];
+
+ } else {
+ ngx_log_stderr(0, "option \"-e\" requires file name");
+ return NGX_ERROR;
+ }
+
+ if (ngx_strcmp(ngx_error_log, "stderr") == 0) {
+ ngx_error_log = (u_char *) "";
+ }
+
+ goto next;
+
case 'c':
if (*p) {
ngx_conf_file = p;
@@ -992,6 +1017,14 @@ ngx_process_options(ngx_cycle_t *cycle)
}
}

+ if (ngx_error_log) {
+ cycle->error_log.len = ngx_strlen(ngx_error_log);
+ cycle->error_log.data = ngx_error_log;
+
+ } else {
+ ngx_str_set(&cycle->error_log, NGX_ERROR_LOG_PATH);
+ }
+
if (ngx_conf_params) {
cycle->conf_param.len = ngx_strlen(ngx_conf_params);
cycle->conf_param.data = ngx_conf_params;
diff -r 4b1299b1856a -r f18db38a9826 src/core/ngx_cycle.c
--- a/src/core/ngx_cycle.c Tue Nov 10 17:13:20 2020 +0300
+++ b/src/core/ngx_cycle.c Thu Nov 19 16:59:00 2020 +0000
@@ -96,6 +96,15 @@ ngx_init_cycle(ngx_cycle_t *old_cycle)
return NULL;
}

+ cycle->error_log.len = old_cycle->error_log.len;
+ cycle->error_log.data = ngx_pnalloc(pool, old_cycle->error_log.len + 1);
+ if (cycle->error_log.data == NULL) {
+ ngx_destroy_pool(pool);
+ return NULL;
+ }
+ ngx_cpystrn(cycle->error_log.data, old_cycle->error_log.data,
+ old_cycle->error_log.len + 1);
+
cycle->conf_file.len = old_cycle->conf_file.len;
cycle->conf_file.data = ngx_pnalloc(pool, old_cycle->conf_file.len + 1);
if (cycle->conf_file.data == NULL) {
diff -r 4b1299b1856a -r f18db38a9826 src/core/ngx_cycle.h
--- a/src/core/ngx_cycle.h Tue Nov 10 17:13:20 2020 +0300
+++ b/src/core/ngx_cycle.h Thu Nov 19 16:59:00 2020 +0000
@@ -80,6 +80,7 @@ struct ngx_cycle_s {
ngx_str_t conf_param;
ngx_str_t conf_prefix;
ngx_str_t prefix;
+ ngx_str_t error_log;
ngx_str_t lock_file;
ngx_str_t hostname;
};
diff -r 4b1299b1856a -r f18db38a9826 src/core/ngx_log.c
--- a/src/core/ngx_log.c Tue Nov 10 17:13:20 2020 +0300
+++ b/src/core/ngx_log.c Thu Nov 19 16:59:00 2020 +0000
@@ -315,7 +315,7 @@ ngx_log_errno(u_char *buf, u_char *last,


ngx_log_t *
-ngx_log_init(u_char *prefix)
+ngx_log_init(u_char *prefix, u_char *error_log)
{
u_char *p, *name;
size_t nlen, plen;
@@ -323,13 +323,11 @@ ngx_log_init(u_char *prefix)
ngx_log.file = &ngx_log_file;
ngx_log.log_level = NGX_LOG_NOTICE;

- name = (u_char *) NGX_ERROR_LOG_PATH;
+ if (error_log == NULL) {
+ error_log = (u_char *) NGX_ERROR_LOG_PATH;
+ }

- /*
- * we use ngx_strlen() here since BCC warns about
- * condition is always false and unreachable code
- */
-
+ name = error_log;
nlen = ngx_strlen(name);

if (nlen == 0) {
@@ -369,7 +367,7 @@ ngx_log_init(u_char *prefix)
*p++ = '/';
}

- ngx_cpystrn(p, (u_char *) NGX_ERROR_LOG_PATH, nlen + 1);
+ ngx_cpystrn(p, error_log, nlen + 1);

p = name;
}
@@ -403,8 +401,7 @@ ngx_log_init(u_char *prefix)
ngx_int_t
ngx_log_open_default(ngx_cycle_t *cycle)
{
- ngx_log_t *log;
- static ngx_str_t error_log = ngx_string(NGX_ERROR_LOG_PATH);
+ ngx_log_t *log;

if (ngx_log_get_file_log(&cycle->new_log) != NULL) {
return NGX_OK;
@@ -425,7 +422,7 @@ ngx_log_open_default(ngx_cycle_t *cycle)

log->log_level = NGX_LOG_ERR;

- log->file = ngx_conf_open_file(cycle, &error_log);
+ log->file = ngx_conf_open_file(cycle, &cycle->error_log);
if (log->file == NULL) {
return NGX_ERROR;
}
diff -r 4b1299b1856a -r f18db38a9826 src/core/ngx_log.h
--- a/src/core/ngx_log.h Tue Nov 10 17:13:20 2020 +0300
+++ b/src/core/ngx_log.h Thu Nov 19 16:59:00 2020 +0000
@@ -228,7 +228,7 @@ void ngx_cdecl ngx_log_debug_core(ngx_lo

/*********************************/

-ngx_log_t *ngx_log_init(u_char *prefix);
+ngx_log_t *ngx_log_init(u_char *prefix, u_char *error_log);
void ngx_cdecl ngx_log_abort(ngx_err_t err, const char *fmt, ...);
void ngx_cdecl ngx_log_stderr(ngx_err_t err, const char *fmt, ...);
u_char *ngx_log_errno(u_char *buf, u_char *last, ngx_err_t err);
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Igor Ippolitov Nginx Development Thu, 19 Nov 2020 12:08:02 -0500
http://www.ldmicj.icu/read.php?29,290004,290004#msg-290004 [njs] HTTP: fixed promise subrequest() with error_page redirect. (no replies) http://www.ldmicj.icu/read.php?29,290004,290004#msg-290004 branches:
changeset: 1568:c947a300b96c
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Wed Nov 18 18:09:11 2020 +0000
description:
HTTP: fixed promise subrequest() with error_page redirect.

Previously, promise callbacks for a subrequest were stored in a
subrequest context. This is incorrect because a subrequest content may
be destroyed (for example when error_page with redirect is enabled for a
subrequest location).

The fix is to store callbacks in the parent context.

The issue was introduced in 6fccbc9f1288 (0.3.8).

diffstat:

nginx/ngx_http_js_module.c | 83 +++++++++++++++++++++++++++++++++++++++------
1 files changed, 72 insertions(+), 11 deletions(-)

diffs (134 lines):

diff -r e97f76121196 -r c947a300b96c nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c Tue Nov 17 13:22:34 2020 +0000
+++ b/nginx/ngx_http_js_module.c Wed Nov 18 18:09:11 2020 +0000
@@ -44,12 +44,18 @@ typedef struct {
njs_opaque_value_t request;
njs_opaque_value_t request_body;
ngx_str_t redirect_uri;
- njs_opaque_value_t promise_callbacks[2];
+ ngx_array_t promise_callbacks;
} ngx_http_js_ctx_t;


typedef struct {
ngx_http_request_t *request;
+ njs_opaque_value_t callbacks[2];
+} ngx_http_js_cb_t;
+
+
+typedef struct {
+ ngx_http_request_t *request;
njs_vm_event_t vm_event;
void *unused;
ngx_int_t ident;
@@ -2276,19 +2282,52 @@ static njs_int_t
ngx_http_js_promise_trampoline(njs_vm_t *vm, njs_value_t *args,
njs_uint_t nargs, njs_index_t unused)
{
+ ngx_uint_t i;
njs_function_t *callback;
+ ngx_http_js_cb_t *cb, *cbs;
ngx_http_js_ctx_t *ctx;
ngx_http_request_t *r;

r = njs_vm_external(vm, njs_argument(args, 1));
- ctx = ngx_http_get_module_ctx(r, ngx_http_js_module);
+ ctx = ngx_http_get_module_ctx(r->parent, ngx_http_js_module);

ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
- "js subrequest promise trampoline ctx: %p", ctx);
-
- callback = njs_value_function(njs_value_arg(&ctx->promise_callbacks[0]));
+ "js subrequest promise trampoline parent ctx: %p", ctx);
+
+ if (ctx == NULL) {
+ njs_vm_error(vm, "js subrequest: failed to get the parent context");
+ return NJS_ERROR;
+ }
+
+ cbs = ctx->promise_callbacks.elts;
+
+ if (cbs == NULL) {
+ goto fail;
+ }
+
+ cb = NULL;
+
+ for (i = 0; i < ctx->promise_callbacks.nelts; i++) {
+ if (cbs[i].request == r) {
+ cb = &cbs[i];
+ cb->request = NULL;
+ break;
+ }
+ }
+
+ if (cb == NULL) {
+ goto fail;
+ }
+
+ callback = njs_value_function(njs_value_arg(&cb->callbacks[0]));

return njs_vm_call(vm, callback, njs_argument(args, 1), 1);
+
+fail:
+
+ njs_vm_error(vm, "js subrequest: promise callback not found");
+
+ return NJS_ERROR;
}


@@ -2298,9 +2337,10 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm,
{
ngx_int_t rc, promise;
njs_str_t uri_arg, args_arg, method_name, body_arg;
- ngx_uint_t method, methods_max, has_body, detached;
+ ngx_uint_t i, method, methods_max, has_body, detached;
njs_value_t *value, *arg, *options;
njs_function_t *callback;
+ ngx_http_js_cb_t *cb, *cbs;
ngx_http_js_ctx_t *ctx;
njs_opaque_value_t lvalue;
ngx_http_request_t *r, *sr;
@@ -2507,15 +2547,36 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm,
}

if (promise) {
- ctx = ngx_pcalloc(sr->pool, sizeof(ngx_http_js_ctx_t));
- if (ctx == NULL) {
- return NGX_ERROR;
+ cbs = ctx->promise_callbacks.elts;
+
+ if (cbs == NULL) {
+ if (ngx_array_init(&ctx->promise_callbacks, r->pool, 4,
+ sizeof(ngx_http_js_cb_t)) != NGX_OK)
+ {
+ goto memory_error;
+ }
}

- ngx_http_set_ctx(sr, ctx, ngx_http_js_module);
+ cb = NULL;
+
+ for (i = 0; i < ctx->promise_callbacks.nelts; i++) {
+ if (cbs[i].request == NULL) {
+ cb = &cbs[i];
+ break;
+ }
+ }
+
+ if (i == ctx->promise_callbacks.nelts) {
+ cb = ngx_array_push(&ctx->promise_callbacks);
+ if (cb == NULL) {
+ goto memory_error;
+ }
+ }
+
+ cb->request = sr;

return njs_vm_promise_create(vm, njs_vm_retval(vm),
- njs_value_arg(&ctx->promise_callbacks));
+ njs_value_arg(&cb->callbacks));
}

njs_value_undefined_set(njs_vm_retval(vm));
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Wed, 18 Nov 2020 13:36:01 -0500
http://www.ldmicj.icu/read.php?29,290001,290001#msg-290001 [njs] Modules: added support for Buffer object where string is expected. (no replies) http://www.ldmicj.icu/read.php?29,290001,290001#msg-290001 branches:
changeset: 1567:e97f76121196
user: Dmitry Volyntsev <xeioex@nginx.com>
date: Tue Nov 17 13:22:34 2020 +0000
description:
Modules: added support for Buffer object where string is expected.

diffstat:

nginx/ngx_http_js_module.c | 2 +-
nginx/ngx_js.c | 2 +-
nginx/ngx_stream_js_module.c | 8 +----
src/njs.h | 6 ++++
src/njs_vm.c | 61 +++++++++++++++++++++++++++++++++++++++++++
src/test/njs_externals_test.c | 8 ++--
6 files changed, 75 insertions(+), 12 deletions(-)

diffs (175 lines):

diff -r 24717ec294b8 -r e97f76121196 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c Tue Nov 17 13:16:47 2020 +0000
+++ b/nginx/ngx_http_js_module.c Tue Nov 17 13:22:34 2020 +0000
@@ -2371,7 +2371,7 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm,
arg = njs_arg(args, nargs, 2);

if (njs_value_is_string(arg)) {
- if (njs_vm_value_to_string(vm, &args_arg, arg) != NJS_OK) {
+ if (ngx_js_string(vm, arg, &args_arg) != NJS_OK) {
njs_vm_error(vm, "failed to convert args");
return NJS_ERROR;
}
diff -r 24717ec294b8 -r e97f76121196 nginx/ngx_js.c
--- a/nginx/ngx_js.c Tue Nov 17 13:16:47 2020 +0000
+++ b/nginx/ngx_js.c Tue Nov 17 13:22:34 2020 +0000
@@ -105,7 +105,7 @@ ngx_int_t
ngx_js_string(njs_vm_t *vm, njs_value_t *value, njs_str_t *str)
{
if (value != NULL && !njs_value_is_null_or_undefined(value)) {
- if (njs_vm_value_to_string(vm, str, value) == NJS_ERROR) {
+ if (njs_vm_value_to_bytes(vm, str, value) == NJS_ERROR) {
return NGX_ERROR;
}

diff -r 24717ec294b8 -r e97f76121196 nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c Tue Nov 17 13:16:47 2020 +0000
+++ b/nginx/ngx_stream_js_module.c Tue Nov 17 13:22:34 2020 +0000
@@ -942,9 +942,7 @@ ngx_stream_js_ext_on(njs_vm_t *vm, njs_v
return NJS_ERROR;
}

- if (njs_vm_value_to_string(vm, &name, njs_arg(args, nargs, 1))
- == NJS_ERROR)
- {
+ if (ngx_js_string(vm, njs_arg(args, nargs, 1), &name) == NJS_ERROR) {
njs_vm_error(vm, "failed to convert event arg");
return NJS_ERROR;
}
@@ -991,9 +989,7 @@ ngx_stream_js_ext_off(njs_vm_t *vm, njs_
return NJS_ERROR;
}

- if (njs_vm_value_to_string(vm, &name, njs_arg(args, nargs, 1))
- == NJS_ERROR)
- {
+ if (ngx_js_string(vm, njs_arg(args, nargs, 1), &name) == NJS_ERROR) {
njs_vm_error(vm, "failed to convert event arg");
return NJS_ERROR;
}
diff -r 24717ec294b8 -r e97f76121196 src/njs.h
--- a/src/njs.h Tue Nov 17 13:16:47 2020 +0000
+++ b/src/njs.h Tue Nov 17 13:22:34 2020 +0000
@@ -339,6 +339,12 @@ NJS_EXPORT njs_int_t njs_vm_value_buffer
const u_char *start, uint32_t size);

/*
+ * Converts a value to bytes.
+ */
+NJS_EXPORT njs_int_t njs_vm_value_to_bytes(njs_vm_t *vm, njs_str_t *dst,
+ njs_value_t *src);
+
+/*
* Converts a value to string.
*/
NJS_EXPORT njs_int_t njs_vm_value_to_string(njs_vm_t *vm, njs_str_t *dst,
diff -r 24717ec294b8 -r e97f76121196 src/njs_vm.c
--- a/src/njs_vm.c Tue Nov 17 13:16:47 2020 +0000
+++ b/src/njs_vm.c Tue Nov 17 13:22:34 2020 +0000
@@ -1116,6 +1116,67 @@ njs_vm_value_to_string(njs_vm_t *vm, njs


njs_int_t
+njs_vm_value_to_bytes(njs_vm_t *vm, njs_str_t *dst, njs_value_t *src)
+{
+ u_char *start;
+ size_t size;
+ njs_int_t ret;
+ njs_value_t value;
+ njs_typed_array_t *array;
+ njs_array_buffer_t *buffer;
+
+ if (njs_slow_path(src == NULL)) {
+ return NJS_ERROR;
+ }
+
+ ret = NJS_OK;
+ value = *src;
+
+ switch (value.type) {
+ case NJS_TYPED_ARRAY:
+ case NJS_DATA_VIEW:
+ array = njs_typed_array(&value);
+ buffer = njs_typed_array_buffer(array);
+ if (njs_slow_path(njs_is_detached_buffer(buffer))) {
+ njs_type_error(vm, "detached buffer");
+ return NJS_ERROR;
+ }
+
+ dst->start = &buffer->u.u8[array->offset];
+ dst->length = array->byte_length;
+ break;
+
+ default:
+ ret = njs_value_to_string(vm, &value, &value);
+ if (njs_slow_path(ret != NJS_OK)) {
+ return NJS_ERROR;
+ }
+
+ size = value.short_string.size;
+
+ if (size != NJS_STRING_LONG) {
+ start = njs_mp_alloc(vm->mem_pool, size);
+ if (njs_slow_path(start == NULL)) {
+ njs_memory_error(vm);
+ return NJS_ERROR;
+ }
+
+ memcpy(start, value.short_string.start, size);
+
+ } else {
+ size = value.long_string.size;
+ start = value.long_string.data->start;
+ }
+
+ dst->length = size;
+ dst->start = start;
+ }
+
+ return ret;
+}
+
+
+njs_int_t
njs_vm_value_string_copy(njs_vm_t *vm, njs_str_t *retval,
njs_value_t *value, uintptr_t *next)
{
diff -r 24717ec294b8 -r e97f76121196 src/test/njs_externals_test.c
--- a/src/test/njs_externals_test.c Tue Nov 17 13:16:47 2020 +0000
+++ b/src/test/njs_externals_test.c Tue Nov 17 13:22:34 2020 +0000
@@ -130,7 +130,7 @@ njs_unit_test_r_uri(njs_vm_t *vm, njs_ob
field = (njs_str_t *) (p + njs_vm_prop_magic32(prop));

if (setval != NULL) {
- return njs_vm_value_to_string(vm, field, setval);
+ return njs_vm_value_to_bytes(vm, field, setval);
}

return njs_vm_value_string_set(vm, retval, field->start, field->length);
@@ -358,7 +358,7 @@ njs_unit_test_r_method(njs_vm_t *vm, njs
return NJS_ERROR;
}

- ret = njs_vm_value_to_string(vm, &s, njs_arg(args, nargs, 1));
+ ret = njs_vm_value_to_bytes(vm, &s, njs_arg(args, nargs, 1));
if (ret == NJS_OK && s.length == 3 && memcmp(s.start, "YES", 3) == 0) {
return njs_vm_value_string_set(vm, njs_vm_retval(vm), r->uri.start,
r->uri.length);
@@ -388,7 +388,7 @@ njs_unit_test_r_create(njs_vm_t *vm, njs
goto memory_error;
}

- if (njs_vm_value_to_string(vm, &sr->uri, njs_arg(args, nargs, 1))
+ if (njs_vm_value_to_bytes(vm, &sr->uri, njs_arg(args, nargs, 1))
!= NJS_OK)
{
return NJS_ERROR;
@@ -424,7 +424,7 @@ njs_unit_test_r_bind(njs_vm_t *vm, njs_v
return NJS_ERROR;
}

- if (njs_vm_value_to_string(vm, &name, njs_arg(args, nargs, 1)) != NJS_OK) {
+ if (njs_vm_value_to_bytes(vm, &name, njs_arg(args, nargs, 1)) != NJS_OK) {
return NJS_ERROR;
}

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Dmitry Volyntsev Nginx Development Wed, 18 Nov 2020 03:34:00 -0500
http://www.ldmicj.icu/read.php?29,289997,289997#msg-289997 [ANN] Test::Nginx 0.29 is released (no replies) http://www.ldmicj.icu/read.php?29,289997,289997#msg-289997
I am happy to announce the new 0.29 release of Test::Nginx:

https://openresty.org/en/ann-test-nginx-029.html

This version fixes the Test2::Util module dependency problem
introduced in the previous 0.28 release.

This Perl module provides a test scaffold for automated testing in
Nginx C module or OpenResty-based Lua library development and
regression testing.

This class inherits from Test::Base, thus bringing all its declarative
power to the Nginx C module testing practices.

All of our OpenResty projects are using this test scaffold for
automated regression testing.

Enjoy!

Best regards,
Yichun
---
Yichun Zhang is the creator of OpenResty, the founder and CEO of OpenResty Inc.
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel]]>
Yichun Zhang (agentzh) Nginx Development Tue, 17 Nov 2020 21:46:02 -0500
Žɳ1005app