Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
If you use or have access to different flavors of FreeBSD, DragonFlyBSD, NetBSD, OpenBSD, Mac OS X, Solaris, or some modern UNIX, I'd appreciate your help.
Some information is sorely lacking in documentation regarding pipe implementation details (except for in Linux), and I'd like to conduct a survey with your help.
Please download, compile, and run the following code:
Download pipesize.c
Language: c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/utsname.h>
#include <errno.h>
#include <limits.h>
void output_system_id()
{
struct utsname n;
if (!uname(&n))
{
char *p = n.release;
unsigned long major, minor, patch;
major = minor = patch = 0;
major = strtoul(p, &p, 10);
if (*p == '.')
{
minor = strtoul(p+1, &p, 10);
if (*p == '.')
{
patch = strtoul(p+1, &p, 10);
}
}
printf("%s %lu.%lu.%lu %s\n", n.sysname, major, minor, patch, n.machine);
}
}
int main()
{
union
{
int pipefd[2];
struct
{
int read;
int write;
} command;
} u;
output_system_id();
printf("PIPE_BUF size: %d\nPipe size: ", PIPE_BUF);
fflush(stdout);
if (!pipe(u.pipefd))
{
int flags = fcntl(u.command.write, F_GETFL, 0);
if (flags != -1)
{
if (fcntl(u.command.write, F_SETFL, flags | O_NONBLOCK) != -1)
{
size_t amount;
for (amount = 0;; amount += sizeof(size_t))
{
ssize_t w = write(u.command.write, &amount, sizeof(size_t));
if (w != sizeof(size_t))
{
if (w == -1)
{
if (errno == EINTR) { amount -= sizeof(size_t); }
else
{
if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) { printf("%zu\n", amount); }
else { perror("Failed to write to pipe"); }
break;
}
}
else if (!w)
{
perror("Pipe unexpectedly closed");
break;
}
else
{
amount -= sizeof(size_t);
amount += w;
}
}
}
}
else { perror("Failed to set pipe nonblocking"); }
}
else { perror("Failed to get pipe flags"); }
close(u.command.read);
close(u.command.write);
}
else { perror("Failed to create pipe"); }
return(0);
}
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I'm quite surprised by that last size... Probably have a bug in my code?
Bisqwit wrote:
Enough?
For Linux yes, Linux is well documented. "man 7 pipe".
Pipe Capacity
A pipe has a limited capacity. If the pipe is full, then a write(2) will block or fail, depending on whether the
O_NONBLOCK flag is set (see below). Different implementations have different limits for the pipe capacity.
Applications should not rely on a particular capacity: an application should be designed so that a reading
process consumes data as soon as it is available, so that a writing process does not remain blocked.
In Linux versions before 2.6.11, the capacity of a pipe was the same as the system page size (e.g., 4096 bytes on
i386). Since Linux 2.6.11, the pipe capacity is 65536 bytes.
PIPE_BUF
POSIX.1-2001 says that write(2)s of less than PIPE_BUF bytes must be atomic: the output data is written to the
pipe as a contiguous sequence. Writes of more than PIPE_BUF bytes may be nonatomic: the kernel may interleave
the data with data written by other processes. POSIX.1-2001 requires PIPE_BUF to be at least 512 bytes. (On
Linux, PIPE_BUF is 4096 bytes.)
The standard is also documented well.
Unfortunately the man pages of other UNIXes out there contain little or no information on their implementation details. So I still want this survey for those to continue.
Thanks for posting Cygwin! :)
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Thanks guys, this is helpful.
Bisqwit: Cygwin not being a power of 2 scares me... Is something wrong somewhere?
I noticed though that recent Cygwin (and recent Linux) has a more direct way of getting the pipe size. Can you try this for me please?
Download pipesize2.c
Language: c
/* Linux 2.6.35+ and recent Cygwin only */
#define _GNU_SOURCE /* Raaaaaaaaaaaaaaaaaaaaaaaaaaaaa */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/utsname.h>
#include <errno.h>
#include <limits.h>
void output_system_id()
{
struct utsname n;
if (!uname(&n))
{
char *p = n.release;
unsigned long major, minor, patch;
major = minor = patch = 0;
major = strtoul(p, &p, 10);
if (*p == '.')
{
minor = strtoul(p+1, &p, 10);
if (*p == '.')
{
patch = strtoul(p+1, &p, 10);
}
}
printf("%s %lu.%lu.%lu %s\n", n.sysname, major, minor, patch, n.machine);
}
}
int main()
{
int pipefd[2];
output_system_id();
printf("PIPE_BUF size: %d\nPipe size: ", PIPE_BUF);
fflush(stdout);
if (!pipe(pipefd))
{
int size = fcntl(*pipefd, F_GETPIPE_SZ, 0);
if (size != -1) { printf("%d\n", size); }
else { perror("Failed to get pipe size"); }
close(pipefd[0]);
close(pipefd[1]);
}
else { perror("Failed to create pipe"); }
return(0);
}
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Not that it counts as a modern unix, but for shits and giggles:
AIX 3.0.0 0003183A4C00
PIPE_BUF size: 32768
Pipe size: 32768
Apparently this version is from 1989? wtf work way to keep up with the joneses
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Thanks Alden, and cool architecture you got there.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Thanks Alden, and cool architecture you got there.
Oh, haha, I didn't even notice that. It's theoretically some IBM PowerPC something or other, I guess?
No idea, Google says nothing on that serial number.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
WST wrote:
Do you still need it?
Linux 3.4.4 i686
PIPE_BUF size: 4096
Pipe size: 65536
I got the info I wanted, thanks.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
