mmap: Add support for MAP_SHARED

[dmik]

Currently, mmap lacks support for MAP_SHARED. This flag is basically ignored and an attempt to use it in another process will cause an error message from WPSTK that this memory doesn't belong to it (this usually is followed by a crash due to access to invalid memory).

MAP_SHARED is often used in Linux software together with MAP_ANONYMOUS (and with fd = -1) to create shared memory for inter-process communication. We need to support that scenario.

[dmik]

Note that MAP_ANONYMOUS works per se. I only had to add a new define for it since MAP_ANON is deprecated now.

I also added a test case from Paul Smedley, anon_mmap.c, that uses the described scenario.

[dmik]

What needs to be done is adding support for shared memory to WPSTK — currently, it only assumes private memory for MMF.

[dmik]

Here's the link to the MMAP manual, for reference: ​http://man7.org/linux/man-pages/man2/mmap.2.html.

[dmik]

I've just discovered that for some strange reason the mman.h header is provided by libc-devel. This is very strange since libc doesn't provide the mmap implementation. This needs to be sorted out too:

• mman.h should be removed from libc-devel
• mman.h from the mmap repo should be installed by mmap-devel

That's all for later.

[Lewis Rosenthal]

Thanks for opening this here, Dmitriy.

As Paul has advised, stable mmap is required for the ​Zend OPcache module for PHP 5.2+ (and now essentially "part" of PHP 5.5).

With more and more web components relying on PHP applications, and such code getting more and more bloated (the latest builds of WordPress offer fine examples of less-than-optimized code), the need to cache as much reusable PHP code in memory has become something more of a necessity than just a nice thing to have. Reports are that the OPcache module should improve performance for things like WordPress by 20% or more.

Much of Arca Noae's web framework will greatly benefit from a working OPcache module.

Please let me know if there is anything I can do on my end to help facilitate prioritizing this work (which I can see is no trivial matter).

Thanks!

[psmedley]

I tried to build gcc with latest mmap support enabled.

Unfortunately it fails with 'virtual memory exhausted' using the path HAVE_MMAP_ANON

Code snippet is:
#ifdef USING_MMAP
/* Allocate SIZE bytes of anonymous memory, preferably near PREF,

(if non-null). The ifdef structure here is intended to cause a
compile error unless exactly one of the HAVE_* is defined. */

static inline char *
alloc_anon (char *pref ATTRIBUTE_UNUSED, size_t size, bool check)
{
#ifdef HAVE_MMAP_ANON

char *page = (char *) mmap (pref, size, PROT_READ | PROT_WRITE,

MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

#endif
#ifdef HAVE_MMAP_DEV_ZERO

char *page = (char *) mmap (pref, size, PROT_READ | PROT_WRITE,

MAP_PRIVATE, G.dev_zero_fd, 0);

#endif

if (page == (char *) MAP_FAILED)

{

if (!check)

return NULL;

perror ("virtual memory exhausted");
exit (FATAL_EXIT_CODE);

}

/* Remember that we allocated this memory. */
G.bytes_mapped += size;

/* Pretend we don't have access to the allocated pages. We'll enable

access to smaller pieces of the area in ggc_internal_alloc. Discard the
handle to avoid handle leak. */

VALGRIND_DISCARD (VALGRIND_MAKE_MEM_NOACCESS (page, size));

return page;

}
#endif

[Silvan Scherrer]

see also #7

[dmik]

BTW, besides adding support for shared memory per se to WPSTK support for LIBC fork() is also needed (as this is how most unix-world programs use mmap).

I'm also considering moving out of WPSTK at all and moving this part of its functionality into LIBCX.

[dmik]

Learning WPSTK mmf machinery shows that it's totally single process oriented. The mmf tables are allocated in private memory and no IPC synchronization. It may really be worth redoing it from scratch within LIBCX.

[dmik]

The LIBCx "redo" ticket is this one: ​https://github.com/bitwiseworks/libcx/issues/11. I've already implemented MAP_SHARED|MAP_ANON case, works like a charm with the anon_mmap.c test case from Paul. So PHP may be rebuilt against libcx.a in order to test it (​http://rpm.netlabs.org/test/tlx1.zip).

Regarding the MAP_PRIVATE|MAP_ANON case, it should also be quite easy to implement (and this is my next task). BTW, the MAP_PRIVATE|MAP_ANON case is essentially a raw malloc with delayed memory allocation, i.e. the initial allocation only reserves address space, physical memory is assigned to it (committed) only when first accessed. This means that such a mmap call may be replaced with a raw malloc in many cases (with the only drawback that it allocates all memory at once instead of as needed). Anyway, I will implement this case and then close this ticket.

[dmik]

This ticket is outdated now as all work has moved to ​​https://github.com/bitwiseworks/libcx/issues/11 (and all functionality mentioned here is already implemented, including dropping the WPSTK dependency). Hence, closing this one.