cregit-Linux how code gets into the kernel

Release 4.11 net/ipv4/tcp.c

Directory: net/ipv4
/*
 * INET         An implementation of the TCP/IP protocol suite for the LINUX
 *              operating system.  INET is implemented using the  BSD Socket
 *              interface as the means of communication with the user level.
 *
 *              Implementation of the Transmission Control Protocol(TCP).
 *
 * Authors:     Ross Biro
 *              Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG>
 *              Mark Evans, <evansmp@uhura.aston.ac.uk>
 *              Corey Minyard <wf-rch!minyard@relay.EU.net>
 *              Florian La Roche, <flla@stud.uni-sb.de>
 *              Charles Hedrick, <hedrick@klinzhai.rutgers.edu>
 *              Linus Torvalds, <torvalds@cs.helsinki.fi>
 *              Alan Cox, <gw4pts@gw4pts.ampr.org>
 *              Matthew Dillon, <dillon@apollo.west.oic.com>
 *              Arnt Gulbrandsen, <agulbra@nvg.unit.no>
 *              Jorge Cwik, <jorge@laser.satlink.net>
 *
 * Fixes:
 *              Alan Cox        :       Numerous verify_area() calls
 *              Alan Cox        :       Set the ACK bit on a reset
 *              Alan Cox        :       Stopped it crashing if it closed while
 *                                      sk->inuse=1 and was trying to connect
 *                                      (tcp_err()).
 *              Alan Cox        :       All icmp error handling was broken
 *                                      pointers passed where wrong and the
 *                                      socket was looked up backwards. Nobody
 *                                      tested any icmp error code obviously.
 *              Alan Cox        :       tcp_err() now handled properly. It
 *                                      wakes people on errors. poll
 *                                      behaves and the icmp error race
 *                                      has gone by moving it into sock.c
 *              Alan Cox        :       tcp_send_reset() fixed to work for
 *                                      everything not just packets for
 *                                      unknown sockets.
 *              Alan Cox        :       tcp option processing.
 *              Alan Cox        :       Reset tweaked (still not 100%) [Had
 *                                      syn rule wrong]
 *              Herp Rosmanith  :       More reset fixes
 *              Alan Cox        :       No longer acks invalid rst frames.
 *                                      Acking any kind of RST is right out.
 *              Alan Cox        :       Sets an ignore me flag on an rst
 *                                      receive otherwise odd bits of prattle
 *                                      escape still
 *              Alan Cox        :       Fixed another acking RST frame bug.
 *                                      Should stop LAN workplace lockups.
 *              Alan Cox        :       Some tidyups using the new skb list
 *                                      facilities
 *              Alan Cox        :       sk->keepopen now seems to work
 *              Alan Cox        :       Pulls options out correctly on accepts
 *              Alan Cox        :       Fixed assorted sk->rqueue->next errors
 *              Alan Cox        :       PSH doesn't end a TCP read. Switched a
 *                                      bit to skb ops.
 *              Alan Cox        :       Tidied tcp_data to avoid a potential
 *                                      nasty.
 *              Alan Cox        :       Added some better commenting, as the
 *                                      tcp is hard to follow
 *              Alan Cox        :       Removed incorrect check for 20 * psh
 *      Michael O'Reilly        :       ack < copied bug fix.
 *      Johannes Stille         :       Misc tcp fixes (not all in yet).
 *              Alan Cox        :       FIN with no memory -> CRASH
 *              Alan Cox        :       Added socket option proto entries.
 *                                      Also added awareness of them to accept.
 *              Alan Cox        :       Added TCP options (SOL_TCP)
 *              Alan Cox        :       Switched wakeup calls to callbacks,
 *                                      so the kernel can layer network
 *                                      sockets.
 *              Alan Cox        :       Use ip_tos/ip_ttl settings.
 *              Alan Cox        :       Handle FIN (more) properly (we hope).
 *              Alan Cox        :       RST frames sent on unsynchronised
 *                                      state ack error.
 *              Alan Cox        :       Put in missing check for SYN bit.
 *              Alan Cox        :       Added tcp_select_window() aka NET2E
 *                                      window non shrink trick.
 *              Alan Cox        :       Added a couple of small NET2E timer
 *                                      fixes
 *              Charles Hedrick :       TCP fixes
 *              Toomas Tamm     :       TCP window fixes
 *              Alan Cox        :       Small URG fix to rlogin ^C ack fight
 *              Charles Hedrick :       Rewrote most of it to actually work
 *              Linus           :       Rewrote tcp_read() and URG handling
 *                                      completely
 *              Gerhard Koerting:       Fixed some missing timer handling
 *              Matthew Dillon  :       Reworked TCP machine states as per RFC
 *              Gerhard Koerting:       PC/TCP workarounds
 *              Adam Caldwell   :       Assorted timer/timing errors
 *              Matthew Dillon  :       Fixed another RST bug
 *              Alan Cox        :       Move to kernel side addressing changes.
 *              Alan Cox        :       Beginning work on TCP fastpathing
 *                                      (not yet usable)
 *              Arnt Gulbrandsen:       Turbocharged tcp_check() routine.
 *              Alan Cox        :       TCP fast path debugging
 *              Alan Cox        :       Window clamping
 *              Michael Riepe   :       Bug in tcp_check()
 *              Matt Dillon     :       More TCP improvements and RST bug fixes
 *              Matt Dillon     :       Yet more small nasties remove from the
 *                                      TCP code (Be very nice to this man if
 *                                      tcp finally works 100%) 8)
 *              Alan Cox        :       BSD accept semantics.
 *              Alan Cox        :       Reset on closedown bug.
 *      Peter De Schrijver      :       ENOTCONN check missing in tcp_sendto().
 *              Michael Pall    :       Handle poll() after URG properly in
 *                                      all cases.
 *              Michael Pall    :       Undo the last fix in tcp_read_urg()
 *                                      (multi URG PUSH broke rlogin).
 *              Michael Pall    :       Fix the multi URG PUSH problem in
 *                                      tcp_readable(), poll() after URG
 *                                      works now.
 *              Michael Pall    :       recv(...,MSG_OOB) never blocks in the
 *                                      BSD api.
 *              Alan Cox        :       Changed the semantics of sk->socket to
 *                                      fix a race and a signal problem with
 *                                      accept() and async I/O.
 *              Alan Cox        :       Relaxed the rules on tcp_sendto().
 *              Yury Shevchuk   :       Really fixed accept() blocking problem.
 *              Craig I. Hagan  :       Allow for BSD compatible TIME_WAIT for
 *                                      clients/servers which listen in on
 *                                      fixed ports.
 *              Alan Cox        :       Cleaned the above up and shrank it to
 *                                      a sensible code size.
 *              Alan Cox        :       Self connect lockup fix.
 *              Alan Cox        :       No connect to multicast.
 *              Ross Biro       :       Close unaccepted children on master
 *                                      socket close.
 *              Alan Cox        :       Reset tracing code.
 *              Alan Cox        :       Spurious resets on shutdown.
 *              Alan Cox        :       Giant 15 minute/60 second timer error
 *              Alan Cox        :       Small whoops in polling before an
 *                                      accept.
 *              Alan Cox        :       Kept the state trace facility since
 *                                      it's handy for debugging.
 *              Alan Cox        :       More reset handler fixes.
 *              Alan Cox        :       Started rewriting the code based on
 *                                      the RFC's for other useful protocol
 *                                      references see: Comer, KA9Q NOS, and
 *                                      for a reference on the difference
 *                                      between specifications and how BSD
 *                                      works see the 4.4lite source.
 *              A.N.Kuznetsov   :       Don't time wait on completion of tidy
 *                                      close.
 *              Linus Torvalds  :       Fin/Shutdown & copied_seq changes.
 *              Linus Torvalds  :       Fixed BSD port reuse to work first syn
 *              Alan Cox        :       Reimplemented timers as per the RFC
 *                                      and using multiple timers for sanity.
 *              Alan Cox        :       Small bug fixes, and a lot of new
 *                                      comments.
 *              Alan Cox        :       Fixed dual reader crash by locking
 *                                      the buffers (much like datagram.c)
 *              Alan Cox        :       Fixed stuck sockets in probe. A probe
 *                                      now gets fed up of retrying without
 *                                      (even a no space) answer.
 *              Alan Cox        :       Extracted closing code better
 *              Alan Cox        :       Fixed the closing state machine to
 *                                      resemble the RFC.
 *              Alan Cox        :       More 'per spec' fixes.
 *              Jorge Cwik      :       Even faster checksumming.
 *              Alan Cox        :       tcp_data() doesn't ack illegal PSH
 *                                      only frames. At least one pc tcp stack
 *                                      generates them.
 *              Alan Cox        :       Cache last socket.
 *              Alan Cox        :       Per route irtt.
 *              Matt Day        :       poll()->select() match BSD precisely on error
 *              Alan Cox        :       New buffers
 *              Marc Tamsky     :       Various sk->prot->retransmits and
 *                                      sk->retransmits misupdating fixed.
 *                                      Fixed tcp_write_timeout: stuck close,
 *                                      and TCP syn retries gets used now.
 *              Mark Yarvis     :       In tcp_read_wakeup(), don't send an
 *                                      ack if state is TCP_CLOSED.
 *              Alan Cox        :       Look up device on a retransmit - routes may
 *                                      change. Doesn't yet cope with MSS shrink right
 *                                      but it's a start!
 *              Marc Tamsky     :       Closing in closing fixes.
 *              Mike Shaver     :       RFC1122 verifications.
 *              Alan Cox        :       rcv_saddr errors.
 *              Alan Cox        :       Block double connect().
 *              Alan Cox        :       Small hooks for enSKIP.
 *              Alexey Kuznetsov:       Path MTU discovery.
 *              Alan Cox        :       Support soft errors.
 *              Alan Cox        :       Fix MTU discovery pathological case
 *                                      when the remote claims no mtu!
 *              Marc Tamsky     :       TCP_CLOSE fix.
 *              Colin (G3TNE)   :       Send a reset on syn ack replies in
 *                                      window but wrong (fixes NT lpd problems)
 *              Pedro Roque     :       Better TCP window handling, delayed ack.
 *              Joerg Reuter    :       No modification of locked buffers in
 *                                      tcp_do_retransmit()
 *              Eric Schenk     :       Changed receiver side silly window
 *                                      avoidance algorithm to BSD style
 *                                      algorithm. This doubles throughput
 *                                      against machines running Solaris,
 *                                      and seems to result in general
 *                                      improvement.
 *      Stefan Magdalinski      :       adjusted tcp_readable() to fix FIONREAD
 *      Willy Konynenberg       :       Transparent proxying support.
 *      Mike McLagan            :       Routing by source
 *              Keith Owens     :       Do proper merging with partial SKB's in
 *                                      tcp_do_sendmsg to avoid burstiness.
 *              Eric Schenk     :       Fix fast close down bug with
 *                                      shutdown() followed by close().
 *              Andi Kleen      :       Make poll agree with SIGIO
 *      Salvatore Sanfilippo    :       Support SO_LINGER with linger == 1 and
 *                                      lingertime == 0 (RFC 793 ABORT Call)
 *      Hirokazu Takahashi      :       Use copy_from_user() instead of
 *                                      csum_and_copy_from_user() if possible.
 *
 *              This program is free software; you can redistribute it and/or
 *              modify it under the terms of the GNU General Public License
 *              as published by the Free Software Foundation; either version
 *              2 of the License, or(at your option) any later version.
 *
 * Description of States:
 *
 *      TCP_SYN_SENT            sent a connection request, waiting for ack
 *
 *      TCP_SYN_RECV            received a connection request, sent ack,
 *                              waiting for final ack in three-way handshake.
 *
 *      TCP_ESTABLISHED         connection established
 *
 *      TCP_FIN_WAIT1           our side has shutdown, waiting to complete
 *                              transmission of remaining buffered data
 *
 *      TCP_FIN_WAIT2           all buffered data sent, waiting for remote
 *                              to shutdown
 *
 *      TCP_CLOSING             both sides have shutdown but we still have
 *                              data we have to finish sending
 *
 *      TCP_TIME_WAIT           timeout to catch resent junk before entering
 *                              closed, can only be entered from FIN_WAIT2
 *                              or CLOSING.  Required because the other end
 *                              may not have gotten our last ACK causing it
 *                              to retransmit the data packet (which we ignore)
 *
 *      TCP_CLOSE_WAIT          remote side has shutdown and is waiting for
 *                              us to finish writing our data and to shutdown
 *                              (we have to close() to move on to LAST_ACK)
 *
 *      TCP_LAST_ACK            out side has shutdown after remote has
 *                              shutdown.  There may still be data in our
 *                              buffer that we have to finish sending
 *
 *      TCP_CLOSE               socket is finished
 */


#define pr_fmt(fmt) "TCP: " fmt

#include <crypto/hash.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/fcntl.h>
#include <linux/poll.h>
#include <linux/inet_diag.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/skbuff.h>
#include <linux/scatterlist.h>
#include <linux/splice.h>
#include <linux/net.h>
#include <linux/socket.h>
#include <linux/random.h>
#include <linux/bootmem.h>
#include <linux/highmem.h>
#include <linux/swap.h>
#include <linux/cache.h>
#include <linux/err.h>
#include <linux/time.h>
#include <linux/slab.h>

#include <net/icmp.h>
#include <net/inet_common.h>
#include <net/tcp.h>
#include <net/xfrm.h>
#include <net/ip.h>
#include <net/sock.h>

#include <linux/uaccess.h>
#include <asm/ioctls.h>
#include <net/busy_poll.h>


int sysctl_tcp_min_tso_segs __read_mostly = 2;


int sysctl_tcp_autocorking __read_mostly = 1;


struct percpu_counter tcp_orphan_count;

EXPORT_SYMBOL_GPL(tcp_orphan_count);


long sysctl_tcp_mem[3] __read_mostly;

int sysctl_tcp_wmem[3] __read_mostly;

int sysctl_tcp_rmem[3] __read_mostly;


EXPORT_SYMBOL(sysctl_tcp_mem);

EXPORT_SYMBOL(sysctl_tcp_rmem);

EXPORT_SYMBOL(sysctl_tcp_wmem);


atomic_long_t tcp_memory_allocated;	
/* Current allocated memory. */

EXPORT_SYMBOL(tcp_memory_allocated);

/*
 * Current number of TCP sockets.
 */

struct percpu_counter tcp_sockets_allocated;

EXPORT_SYMBOL(tcp_sockets_allocated);

/*
 * TCP splice context
 */

struct tcp_splice_state {
	
struct pipe_inode_info *pipe;
	
size_t len;
	
unsigned int flags;
};

/*
 * Pressure flag: try to collapse.
 * Technical note: it is used by multiple contexts non atomically.
 * All the __sk_mem_schedule() is of this nature: accounting
 * is strict, actions are advisory and have some latency.
 */

int tcp_memory_pressure __read_mostly;

EXPORT_SYMBOL(tcp_memory_pressure);


void tcp_enter_memory_pressure(struct sock *sk) { if (!tcp_memory_pressure) { NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMEMORYPRESSURES); tcp_memory_pressure = 1; } }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)1548.39%116.67%
Pavel Emelyanov929.03%233.33%
Arnaldo Carvalho de Melo619.35%233.33%
Hideaki Yoshifuji / 吉藤英明13.23%116.67%
Total31100.00%6100.00%

EXPORT_SYMBOL(tcp_enter_memory_pressure); /* Convert seconds to retransmits based on initial and max timeout */
static u8 secs_to_retrans(int seconds, int timeout, int rto_max) { u8 res = 0; if (seconds > 0) { int period = timeout; res = 1; while (seconds > period && res < 255) { res++; timeout <<= 1; if (timeout > rto_max) timeout = rto_max; period += timeout; } } return res; }

Contributors

PersonTokensPropCommitsCommitProp
Julian Anastasov73100.00%1100.00%
Total73100.00%1100.00%

/* Convert retransmits to seconds based on initial and max timeout */
static int retrans_to_secs(u8 retrans, int timeout, int rto_max) { int period = 0; if (retrans > 0) { period = timeout; while (--retrans) { timeout <<= 1; if (timeout > rto_max) timeout = rto_max; period += timeout; } } return period; }

Contributors

PersonTokensPropCommitsCommitProp
Julian Anastasov60100.00%1100.00%
Total60100.00%1100.00%

/* Address-family independent initialization for a tcp_sock. * * NOTE: A lot of things set to zero explicitly by call to * sk_alloc() so need not be done here. */
void tcp_init_sock(struct sock *sk) { struct inet_connection_sock *icsk = inet_csk(sk); struct tcp_sock *tp = tcp_sk(sk); tp->out_of_order_queue = RB_ROOT; tcp_init_xmit_timers(sk); tcp_prequeue_init(tp); INIT_LIST_HEAD(&tp->tsq_node); icsk->icsk_rto = TCP_TIMEOUT_INIT; tp->mdev_us = jiffies_to_usecs(TCP_TIMEOUT_INIT); minmax_reset(&tp->rtt_min, tcp_time_stamp, ~0U); /* So many TCP implementations out there (incorrectly) count the * initial SYN frame in their delayed-ACK and congestion control * algorithms that we must have the following bandaid to talk * efficiently to them. -DaveM */ tp->snd_cwnd = TCP_INIT_CWND; /* There's a bubble in the pipe until at least the first ACK. */ tp->app_limited = ~0U; /* See draft-stevens-tcpca-spec-01 for discussion of the * initialization of these values. */ tp->snd_ssthresh = TCP_INFINITE_SSTHRESH; tp->snd_cwnd_clamp = ~0; tp->mss_cache = TCP_MSS_DEFAULT; tp->reordering = sock_net(sk)->ipv4.sysctl_tcp_reordering; tcp_assign_congestion_control(sk); tp->tsoffset = 0; sk->sk_state = TCP_CLOSE; sk->sk_write_space = sk_stream_write_space; sock_set_flag(sk, SOCK_USE_WRITE_QUEUE); icsk->icsk_sync_mss = tcp_sync_mss; sk->sk_sndbuf = sysctl_tcp_wmem[1]; sk->sk_rcvbuf = sysctl_tcp_rmem[1]; sk_sockets_allocated_inc(sk); }

Contributors

PersonTokensPropCommitsCommitProp
Neal Cardwell14476.19%220.00%
Eric Dumazet126.35%220.00%
Soheil Hassas Yeganeh84.23%110.00%
Nikolay Borisov73.70%110.00%
Yuchung Cheng63.17%110.00%
Andrey Vagin63.17%110.00%
Florian Westphal42.12%110.00%
Yaogong Wang21.06%110.00%
Total189100.00%10100.00%

EXPORT_SYMBOL(tcp_init_sock);
static void tcp_tx_timestamp(struct sock *sk, u16 tsflags, struct sk_buff *skb) { if (tsflags && skb) { struct skb_shared_info *shinfo = skb_shinfo(skb); struct tcp_skb_cb *tcb = TCP_SKB_CB(skb); sock_tx_timestamp(sk, tsflags, &shinfo->tx_flags); if (tsflags & SOF_TIMESTAMPING_TX_ACK) tcb->txstamp_ack = 1; if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK) shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; } }

Contributors

PersonTokensPropCommitsCommitProp
Willem de Bruijn6367.02%233.33%
Soheil Hassas Yeganeh3132.98%466.67%
Total94100.00%6100.00%

/* * Wait for a TCP event. * * Note that we don't need to lock the socket, as the upper poll layers * take care of normal races (between the test and the event) and we don't * go look at any of the socket buffers directly. */
unsigned int tcp_poll(struct file *file, struct socket *sock, poll_table *wait) { unsigned int mask; struct sock *sk = sock->sk; const struct tcp_sock *tp = tcp_sk(sk); int state; sock_rps_record_flow(sk); sock_poll_wait(file, sk_sleep(sk), wait); state = sk_state_load(sk); if (state == TCP_LISTEN) return inet_csk_listen_poll(sk); /* Socket is not locked. We are protected from async events * by poll logic and correct handling of state changes * made by other threads is impossible in any case. */ mask = 0; /* * POLLHUP is certainly not done right. But poll() doesn't * have a notion of HUP in just one direction, and for a * socket the read side is more interesting. * * Some poll() documentation says that POLLHUP is incompatible * with the POLLOUT/POLLWR flags, so somebody should check this * all. But careful, it tends to be safer to return too many * bits than too few, and you can easily break real applications * if you don't tell them that something has hung up! * * Check-me. * * Check number 1. POLLHUP is _UNMASKABLE_ event (see UNIX98 and * our fs/select.c). It means that after we received EOF, * poll always returns immediately, making impossible poll() on write() * in state CLOSE_WAIT. One solution is evident --- to set POLLHUP * if and only if shutdown has been made in both directions. * Actually, it is interesting to look how Solaris and DUX * solve this dilemma. I would prefer, if POLLHUP were maskable, * then we could set it on SND_SHUTDOWN. BTW examples given * in Stevens' books assume exactly this behaviour, it explains * why POLLHUP is incompatible with POLLOUT. --ANK * * NOTE. Check for TCP_CLOSE is added. The goal is to prevent * blocking on fresh not-connected or disconnected socket. --ANK */ if (sk->sk_shutdown == SHUTDOWN_MASK || state == TCP_CLOSE) mask |= POLLHUP; if (sk->sk_shutdown & RCV_SHUTDOWN) mask |= POLLIN | POLLRDNORM | POLLRDHUP; /* Connected or passive Fast Open socket? */ if (state != TCP_SYN_SENT && (state != TCP_SYN_RECV || tp->fastopen_rsk)) { int target = sock_rcvlowat(sk, 0, INT_MAX); if (tp->urg_seq == tp->copied_seq && !sock_flag(sk, SOCK_URGINLINE) && tp->urg_data) target++; if (tp->rcv_nxt - tp->copied_seq >= target) mask |= POLLIN | POLLRDNORM; if (!(sk->sk_shutdown & SEND_SHUTDOWN)) { if (sk_stream_is_writeable(sk)) { mask |= POLLOUT | POLLWRNORM; } else { /* send SIGIO later */ sk_set_bit(SOCKWQ_ASYNC_NOSPACE, sk); set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); /* Race breaker. If space is freed after * wspace test but before the flags are set, * IO signal will be lost. Memory barrier * pairs with the input side. */ smp_mb__after_atomic(); if (sk_stream_is_writeable(sk)) mask |= POLLOUT | POLLWRNORM; } } else mask |= POLLOUT | POLLWRNORM; if (tp->urg_data & TCP_URG_VALID) mask |= POLLPRI; } else if (sk->sk_state == TCP_SYN_SENT && inet_sk(sk)->defer_connect) { /* Active TCP fastopen socket with defer_connect * Return POLLOUT so application can call write() * in order for kernel to generate SYN+data */ mask |= POLLOUT | POLLWRNORM; } /* This barrier is coupled with smp_wmb() in tcp_reset() */ smp_rmb(); if (sk->sk_err || !skb_queue_empty(&sk->sk_error_queue)) mask |= POLLERR; return mask; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)18655.86%2552.08%
David S. Miller339.91%24.17%
Wei Wang257.51%12.08%
Eric Dumazet226.61%510.42%
Tom Marshall144.20%12.08%
Jerry Chu103.00%12.08%
Arnaldo Carvalho de Melo103.00%48.33%
Willem de Bruijn92.70%12.08%
Motohiro Kosaki72.10%12.08%
David Majnemer51.50%12.08%
Jason Baron41.20%12.08%
James Morris20.60%12.08%
Davide Libenzi20.60%12.08%
Will Newton20.60%12.08%
Jiri Olsa10.30%12.08%
Alexandra N. Kossovsky10.30%12.08%
Total333100.00%48100.00%

EXPORT_SYMBOL(tcp_poll);
int tcp_ioctl(struct sock *sk, int cmd, unsigned long arg) { struct tcp_sock *tp = tcp_sk(sk); int answ; bool slow; switch (cmd) { case SIOCINQ: if (sk->sk_state == TCP_LISTEN) return -EINVAL; slow = lock_sock_fast(sk); answ = tcp_inq(sk); unlock_sock_fast(sk, slow); break; case SIOCATMARK: answ = tp->urg_data && tp->urg_seq == tp->copied_seq; break; case SIOCOUTQ: if (sk->sk_state == TCP_LISTEN) return -EINVAL; if ((1 << sk->sk_state) & (TCPF_SYN_SENT | TCPF_SYN_RECV)) answ = 0; else answ = tp->write_seq - tp->snd_una; break; case SIOCOUTQNSD: if (sk->sk_state == TCP_LISTEN) return -EINVAL; if ((1 << sk->sk_state) & (TCPF_SYN_SENT | TCPF_SYN_RECV)) answ = 0; else answ = tp->write_seq - tp->snd_nxt; break; default: return -ENOIOCTLCMD; } return put_user(answ, (int __user *)arg); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)13866.99%110.00%
Mario Schuknecht4722.82%110.00%
Eric Dumazet104.85%220.00%
Arnaldo Carvalho de Melo62.91%330.00%
David S. Miller31.46%110.00%
Tom Herbert10.49%110.00%
Al Viro10.49%110.00%
Total206100.00%10100.00%

EXPORT_SYMBOL(tcp_ioctl);
static inline void tcp_mark_push(struct tcp_sock *tp, struct sk_buff *skb) { TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_PSH; tp->pushed_seq = tp->write_seq; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds3191.18%125.00%
Eric Dumazet12.94%125.00%
Arnaldo Carvalho de Melo12.94%125.00%
Changli Gao12.94%125.00%
Total34100.00%4100.00%


static inline bool forced_push(const struct tcp_sock *tp) { return after(tp->write_seq, tp->pushed_seq + (tp->max_window >> 1)); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds3090.91%125.00%
Eric Dumazet26.06%250.00%
Arnaldo Carvalho de Melo13.03%125.00%
Total33100.00%4100.00%


static void skb_entail(struct sock *sk, struct sk_buff *skb) { struct tcp_sock *tp = tcp_sk(sk); struct tcp_skb_cb *tcb = TCP_SKB_CB(skb); skb->csum = 0; tcb->seq = tcb->end_seq = tp->write_seq; tcb->tcp_flags = TCPHDR_ACK; tcb->sacked = 0; __skb_header_release(skb); tcp_add_write_queue_tail(sk, skb); sk->sk_wmem_queued += skb->truesize; sk_mem_charge(sk, skb->truesize); if (tp->nonagle & TCP_NAGLE_PUSH) tp->nonagle &= ~TCP_NAGLE_PUSH; tcp_slow_start_after_idle_check(sk); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds5446.96%19.09%
Arnaldo Carvalho de Melo1412.17%19.09%
Alexey Kuznetsov1311.30%19.09%
Hideo Aoki119.57%19.09%
Ilpo Järvinen108.70%19.09%
Eric Dumazet76.09%327.27%
Herbert Xu43.48%19.09%
David S. Miller10.87%19.09%
Changli Gao10.87%19.09%
Total115100.00%11100.00%


static inline void tcp_mark_urg(struct tcp_sock *tp, int flags) { if (flags & MSG_OOB) tp->snd_up = tp->write_seq; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds2896.55%150.00%
Arnaldo Carvalho de Melo13.45%150.00%
Total29100.00%2100.00%

/* If a not yet filled skb is pushed, do not send it if * we have data packets in Qdisc or NIC queues : * Because TX completion will happen shortly, it gives a chance * to coalesce future sendmsg() payload into this skb, without * need for a timer, and with no latency trade off. * As packets containing data payload have a bigger truesize * than pure acks (dataless) packets, the last checks prevent * autocorking if we only have an ACK in Qdisc/NIC queues, * or if TX completion was delayed after we processed ACK packet. */
static bool tcp_should_autocork(struct sock *sk, struct sk_buff *skb, int size_goal) { return skb->len < size_goal && sysctl_tcp_autocorking && skb != tcp_write_queue_head(sk) && atomic_read(&sk->sk_wmem_alloc) > skb->truesize; }

Contributors

PersonTokensPropCommitsCommitProp
Eric Dumazet47100.00%2100.00%
Total47100.00%2100.00%


static void tcp_push(struct sock *sk, int flags, int mss_now, int nonagle, int size_goal) { struct tcp_sock *tp = tcp_sk(sk); struct sk_buff *skb; if (!tcp_send_head(sk)) return; skb = tcp_write_queue_tail(sk); if (!(flags & MSG_MORE) || forced_push(tp)) tcp_mark_push(tp, skb); tcp_mark_urg(tp, flags); if (tcp_should_autocork(sk, skb, size_goal)) { /* avoid atomic op if TSQ_THROTTLED bit is already set */ if (!test_bit(TSQ_THROTTLED, &sk->sk_tsq_flags)) { NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAUTOCORKING); set_bit(TSQ_THROTTLED, &sk->sk_tsq_flags); } /* It is possible TX completion already happened * before we set TSQ_THROTTLED. */ if (atomic_read(&sk->sk_wmem_alloc) > skb->truesize) return; } if (flags & MSG_MORE) nonagle = TCP_NAGLE_CORK; __tcp_push_pending_frames(sk, mss_now, nonagle); }

Contributors

PersonTokensPropCommitsCommitProp
Eric Dumazet9859.04%342.86%
Linus Torvalds6237.35%114.29%
Krishna Kumar31.81%114.29%
David S. Miller21.20%114.29%
Alexey Kuznetsov10.60%114.29%
Total166100.00%7100.00%


static int tcp_splice_data_recv(read_descriptor_t *rd_desc, struct sk_buff *skb, unsigned int offset, size_t len) { struct tcp_splice_state *tss = rd_desc->arg.data; int ret; ret = skb_splice_bits(skb, skb->sk, offset, tss->pipe, min(rd_desc->count, len), tss->flags); if (ret > 0) rd_desc->count -= ret; return ret; }

Contributors

PersonTokensPropCommitsCommitProp
Jens Axboe4960.49%120.00%
Willy Tarreau2227.16%120.00%
Dimitris Michailidis56.17%120.00%
Hannes Frederic Sowa44.94%120.00%
Adrian Bunk11.23%120.00%
Total81100.00%5100.00%


static int __tcp_splice_read(struct sock *sk, struct tcp_splice_state *tss) { /* Store TCP splice context information in read_descriptor_t. */ read_descriptor_t rd_desc = { .arg.data = tss, .count = tss->len, }; return tcp_read_sock(sk, &rd_desc, tcp_splice_data_recv); }

Contributors

PersonTokensPropCommitsCommitProp
Jens Axboe4085.11%150.00%
Willy Tarreau714.89%150.00%
Total47100.00%2100.00%

/** * tcp_splice_read - splice data from TCP socket to a pipe * @sock: socket to splice from * @ppos: position (not valid) * @pipe: pipe to splice to * @len: number of bytes to splice * @flags: splice modifier flags * * Description: * Will read pages from given socket and fill them into a pipe. * **/
ssize_t tcp_splice_read(struct socket *sock, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags) { struct sock *sk = sock->sk; struct tcp_splice_state tss = { .pipe = pipe, .len = len, .flags = flags, }; long timeo; ssize_t spliced; int ret; sock_rps_record_flow(sk); /* * We can't seek on a socket input */ if (unlikely(*ppos)) return -ESPIPE; ret = spliced = 0; lock_sock(sk); timeo = sock_rcvtimeo(sk, sock->file->f_flags & O_NONBLOCK); while (tss.len) { ret = __tcp_splice_read(sk, &tss); if (ret < 0) break; else if (!ret) { if (spliced) break; if (sock_flag(sk, SOCK_DONE)) break; if (sk->sk_err) { ret = sock_error(sk); break; } if (sk->sk_shutdown & RCV_SHUTDOWN) break; if (sk->sk_state == TCP_CLOSE) { /* * This occurs when user tries to read * from never connected socket. */ if (!sock_flag(sk, SOCK_DONE)) ret = -ENOTCONN; break; } if (!timeo) { ret = -EAGAIN; break; } /* if __tcp_splice_read() got nothing while we have * an skb in receive queue, we do not want to loop. * This might happen with URG data. */ if (!skb_queue_empty(&sk->sk_receive_queue)) break; sk_wait_data(sk, &timeo, NULL); if (signal_pending(current)) { ret = sock_intr_errno(timeo); break; } continue; } tss.len -= ret; spliced += ret; if (!timeo) break; release_sock(sk); lock_sock(sk); if (sk->sk_err || sk->sk_state == TCP_CLOSE || (sk->sk_shutdown & RCV_SHUTDOWN) || signal_pending(current)) break; } release_sock(sk); if (spliced) return spliced; return ret; }

Contributors

PersonTokensPropCommitsCommitProp
Jens Axboe29890.30%116.67%
Eric Dumazet195.76%233.33%
Willy Tarreau61.82%116.67%
Changli Gao51.52%116.67%
Sabrina Dubroca20.61%116.67%
Total330100.00%6100.00%

EXPORT_SYMBOL(tcp_splice_read);
struct sk_buff *sk_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp, bool force_schedule) { struct sk_buff *skb; /* The TCP header must be at least 32-bit aligned. */ size = ALIGN(size, 4); if (unlikely(tcp_under_memory_pressure(sk))) sk_mem_reclaim_partial(sk); skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); if (likely(skb)) { bool mem_scheduled; if (force_schedule) { mem_scheduled = true; sk_forced_mem_schedule(sk, skb->truesize); } else { mem_scheduled = sk_wmem_schedule(sk, skb->truesize); } if (likely(mem_scheduled)) { skb_reserve(skb, sk->sk_prot->max_header); /* * Make sure that we have exactly size bytes * available to the caller, no more, no less. */ skb->reserved_tailroom = skb->end - skb->tail - size; return skb; } __kfree_skb(skb); } else { sk->sk_prot->enter_memory_pressure(sk); sk_stream_moderate_sndbuf(sk); } return NULL; }

Contributors

PersonTokensPropCommitsCommitProp
Pavel Emelyanov10158.05%337.50%
Eric Dumazet7241.38%450.00%
Hideo Aoki10.57%112.50%
Total174100.00%8100.00%


static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, int large_allowed) { struct tcp_sock *tp = tcp_sk(sk); u32 new_size_goal, size_goal; if (!large_allowed || !sk_can_gso(sk)) return mss_now; /* Note : tcp_tso_autosize() will eventually split this later */ new_size_goal = sk->sk_gso_max_size - 1 - MAX_TCP_HEADER; new_size_goal = tcp_bound_to_half_wnd(tp, new_size_goal); /* We try hard to avoid divides here */ size_goal = tp->gso_segs * mss_now; if (unlikely(new_size_goal < size_goal || new_size_goal >= size_goal + mss_now)) { tp->gso_segs = min_t(u16, new_size_goal / mss_now, sk->sk_gso_max_segs); size_goal = tp->gso_segs * mss_now; } return max(size_goal, mss_now); }

Contributors

PersonTokensPropCommitsCommitProp
Ilpo Järvinen8164.29%342.86%
Eric Dumazet3628.57%342.86%
Ben Hutchings97.14%114.29%
Total126100.00%7100.00%


static int tcp_send_mss(struct sock *sk, int *size_goal, int flags) { int mss_now; mss_now = tcp_current_mss(sk); *size_goal = tcp_xmit_size_goal(sk, mss_now, !(flags & MSG_OOB)); return mss_now; }

Contributors

PersonTokensPropCommitsCommitProp
Ilpo Järvinen48100.00%1100.00%
Total48100.00%1100.00%


static ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, size_t size, int flags) { struct tcp_sock *tp = tcp_sk(sk); int mss_now, size_goal; int err; ssize_t copied; long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); /* Wait for a connection to finish. One exception is TCP Fast Open * (passive side) where data is allowed to be sent before a connection * is fully established. */ if (((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) && !tcp_passive_fastopen(sk)) { err = sk_stream_wait_connect(sk, &timeo); if (err != 0) goto out_err; } sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); mss_now = tcp_send_mss(sk, &size_goal, flags); copied = 0; err = -EPIPE; if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) goto out_err; while (size > 0) { struct sk_buff *skb = tcp_write_queue_tail(sk); int copy, i; bool can_coalesce; if (!tcp_send_head(sk) || (copy = size_goal - skb->len) <= 0 || !tcp_skb_can_collapse_to(skb)) { new_segment: if (!sk_stream_memory_free(sk)) goto wait_for_sndbuf; skb = sk_stream_alloc_skb(sk, 0, sk->sk_allocation, skb_queue_empty(&sk->sk_write_queue)); if (!skb) goto wait_for_memory; skb_entail(sk, skb); copy = size_goal; } if (copy > size) copy = size; i = skb_shinfo(skb)->nr_frags; can_coalesce = skb_can_coalesce(skb, i, page, offset); if (!can_coalesce && i >= sysctl_max_skb_frags) { tcp_mark_push(tp, skb); goto new_segment; } if (!sk_wmem_schedule(sk, copy)) goto wait_for_memory; if (can_coalesce) { skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); } else { get_page(page); skb_fill_page_desc(skb, i, page, offset, copy); } skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG; skb->len += copy; skb->data_len += copy; skb->truesize += copy; sk->sk_wmem_queued += copy; sk_mem_charge(sk, copy); skb->ip_summed = CHECKSUM_PARTIAL; tp->write_seq += copy; TCP_SKB_CB(skb)->end_seq += copy; tcp_skb_pcount_set(skb, 0); if (!copied) TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH; copied += copy; offset += copy; size -= copy; if (!size) goto out; if (skb->len < size_goal || (flags & MSG_OOB)) continue; if (forced_push(tp)) { tcp_mark_push(tp, skb); __tcp_push_pending_frames(sk, mss_now, TCP_NAGLE_PUSH); } else if (skb == tcp_send_head(sk)) tcp_push_one(sk, mss_now); continue; wait_for_sndbuf: set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); wait_for_memory: tcp_push(sk, flags & ~MSG_MORE, mss_now, TCP_NAGLE_PUSH, size_goal); err = sk_stream_wait_memory(sk, &timeo); if (err != 0) goto do_error; mss_now = tcp_send_mss(sk, &size_goal, flags); } out: if (copied) { tcp_tx_timestamp(sk, sk->sk_tsflags, tcp_write_queue_tail(sk)); if (!(flags & MSG_SENDPAGE_NOTLAST)) tcp_push(sk, flags, mss_now, tp->nonagle, size_goal); } return copied; do_error: if (copied) goto out; out_err: /* make sure we wake any epoll edge trigger waiter */ if (unlikely(skb_queue_len(&sk->sk_write_queue) == 0 && err == -EAGAIN)) { sk->sk_write_space(sk); tcp_chrono_stop(sk, TCP_CHRONO_SNDBUF_LIMITED); } return sk_stream_error(sk, flags, err); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds44163.09%12.17%
David S. Miller7911.30%613.04%
Eric Dumazet456.44%1123.91%
Jason Baron284.01%12.17%
Arnaldo Carvalho de Melo213.00%1021.74%
Soheil Hassas Yeganeh192.72%12.17%
Yuvaraja Mariappan121.72%12.17%
Jerry Chu111.57%12.17%
Ilpo Järvinen91.29%24.35%
Francis Yan91.29%12.17%
Martin KaFai Lau60.86%12.17%
Alexey Kuznetsov60.86%24.35%
Hideo Aoki50.72%12.17%
Pravin B Shelar20.29%12.17%
Hans Westgaard Ry10.14%12.17%
Patrick McHardy10.14%12.17%
Stephen Hemminger10.14%12.17%
Pavel Emelyanov10.14%12.17%
Changli Gao10.14%12.17%
Herbert Xu10.14%12.17%
Total699100.00%46100.00%


int tcp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { ssize_t res; if (!(sk->sk_route_caps & NETIF_F_SG) || !sk_check_csum_caps(sk)) return sock_no_sendpage(sk->sk_socket, page, offset, size, flags); lock_sock(sk); tcp_rate_check_app_limited(sk); /* is sending application-limited? */ res = do_tcp_sendpages(sk, page, offset, size, flags); release_sock(sk); return res; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds7882.98%120.00%
Changli Gao66.38%120.00%
Soheil Hassas Yeganeh66.38%120.00%
Tom Herbert33.19%120.00%
Arnaldo Carvalho de Melo11.06%120.00%
Total94100.00%5100.00%

EXPORT_SYMBOL(tcp_sendpage); /* Do not bother using a page frag for very small frames. * But use this heuristic only for the first skb in write queue. * * Having no payload in skb->head allows better SACK shifting * in tcp_shift_skb_data(), reducing sack/rack overhead, because * write queue has less skbs. * Each skb can hold up to MAX_SKB_FRAGS * 32Kbytes, or ~0.5 MB. * This also speeds up tso_fragment(), since it wont fallback * to tcp_fragment(). */
static int linear_payload_sz(bool first_skb) { if (first_skb) return SKB_WITH_OVERHEAD(2048 - MAX_TCP_HEADER); return 0; }

Contributors

PersonTokensPropCommitsCommitProp
Eric Dumazet24100.00%1100.00%
Total24100.00%1100.00%


static int select_size(const struct sock *sk, bool sg, bool first_skb) { const struct tcp_sock *tp = tcp_sk(sk); int tmp = tp->mss_cache; if (sg) { if (sk_can_gso(sk)) { tmp = linear_payload_sz(first_skb); } else { int pgbreak = SKB_MAX_HEAD(MAX_TCP_HEADER); if (tmp >= pgbreak && tmp <= pgbreak + (MAX_SKB_FRAGS - 1) * PAGE_SIZE) tmp = pgbreak; } } return tmp; }

Contributors

PersonTokensPropCommitsCommitProp
David S. Miller4042.55%218.18%
Linus Torvalds2728.72%19.09%
Eric Dumazet1212.77%436.36%
Ilpo Järvinen88.51%19.09%
Herbert Xu33.19%19.09%
Krishna Kumar33.19%19.09%
Arnaldo Carvalho de Melo11.06%19.09%
Total94100.00%11100.00%


void tcp_free_fastopen_req(struct tcp_sock *tp) { if (tp->fastopen_req) { kfree(tp->fastopen_req); tp->fastopen_req = NULL; } }

Contributors

PersonTokensPropCommitsCommitProp
Yuchung Cheng31100.00%1100.00%
Total31100.00%1100.00%


static int tcp_sendmsg_fastopen(struct sock *sk, struct msghdr *msg, int *copied, size_t size) { struct tcp_sock *tp = tcp_sk(sk); struct inet_sock *inet = inet_sk(sk); int err, flags; if (!(sysctl_tcp_fastopen & TFO_CLIENT_ENABLE)) return -EOPNOTSUPP; if (tp->fastopen_req) return -EALREADY; /* Another Fast Open is in progress */ tp->fastopen_req = kzalloc(sizeof(struct tcp_fastopen_request), sk->sk_allocation); if (unlikely(!tp->fastopen_req)) return -ENOBUFS; tp->fastopen_req->data = msg; tp->fastopen_req->size = size; if (inet->defer_connect) { err = tcp_connect(sk); /* Same failure procedure as in tcp_v4/6_connect */ if (err) { tcp_set_state(sk, TCP_CLOSE); inet->inet_dport = 0; sk->sk_route_caps = 0; } } flags = (msg->msg_flags & MSG_DONTWAIT) ? O_NONBLOCK : 0; err = __inet_stream_connect(sk->sk_socket, msg->msg_name, msg->msg_namelen, flags, 1); /* fastopen_req could already be freed in __inet_stream_connect * if the connection times out or gets rst */ if (tp->fastopen_req) { *copied = tp->fastopen_req->copied; tcp_free_fastopen_req(tp); inet->defer_connect = 0; } return err; }

Contributors

PersonTokensPropCommitsCommitProp
Yuchung Cheng14664.32%116.67%
Wei Wang6629.07%233.33%
Eric Dumazet125.29%116.67%
Willy Tarreau20.88%116.67%
Ian Morris10.44%116.67%
Total227100.00%6100.00%


int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { struct tcp_sock *tp = tcp_sk(sk); struct sk_buff *skb; struct sockcm_cookie sockc; int flags, err, copied = 0; int mss_now = 0, size_goal, copied_syn = 0; bool process_backlog = false; bool sg; long timeo; lock_sock(sk); flags = msg->msg_flags; if (unlikely(flags & MSG_FASTOPEN || inet_sk(sk)->defer_connect)) { err = tcp_sendmsg_fastopen(sk, msg, &copied_syn, size); if (err == -EINPROGRESS && copied_syn > 0) goto out; else if (err) goto out_err; } timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); tcp_rate_check_app_limited(sk); /* is sending application-limited? */ /* Wait for a connection to finish. One exception is TCP Fast Open * (passive side) where data is allowed to be sent before a connection * is fully established. */ if (((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) && !tcp_passive_fastopen(sk)) { err = sk_stream_wait_connect(sk, &timeo); if (err != 0) goto do_error; } if (unlikely(tp->repair)) { if (tp->repair_queue == TCP_RECV_QUEUE) { copied = tcp_send_rcvq(sk, msg, size); goto out_nopush; } err = -EINVAL; if (tp->repair_queue == TCP_NO_QUEUE) goto out_err; /* 'common' sending to sendq */ } sockc.tsflags = sk->sk_tsflags; if (msg->msg_controllen) { err = sock_cmsg_send(sk, msg, &sockc); if (unlikely(err)) { err = -EINVAL; goto out_err; } } /* This should be in poll */ sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); /* Ok commence sending. */ copied = 0; restart: mss_now = tcp_send_mss(sk, &size_goal, flags); err = -EPIPE; if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) goto do_error; sg = !!(sk->sk_route_caps & NETIF_F_SG); while (msg_data_left(msg)) { int copy = 0; int max = size_goal; skb = tcp_write_queue_tail(sk); if (tcp_send_head(sk)) { if (skb->ip_summed == CHECKSUM_NONE) max = mss_now; copy = max - skb->len; } if (copy <= 0 || !tcp_skb_can_collapse_to(skb)) { bool first_skb; new_segment: /* Allocate new segment. If the interface is SG, * allocate skb fitting to single page. */ if (!sk_stream_memory_free(sk)) goto wait_for_sndbuf; if (process_backlog && sk_flush_backlog(sk)) { process_backlog = false; goto restart; } first_skb = skb_queue_empty(&sk->sk_write_queue); skb = sk_stream_alloc_skb(sk, select_size(sk, sg, first_skb), sk->sk_allocation, first_skb); if (!skb) goto wait_for_memory; process_backlog = true; /* * Check whether we can use HW checksum. */ if (sk_check_csum_caps(sk)) skb->ip_summed = CHECKSUM_PARTIAL; skb_entail(sk, skb); copy = size_goal; max = size_goal; /* All packets are restored as if they have * already been sent. skb_mstamp isn't set to * avoid wrong rtt estimation. */ if (tp->repair) TCP_SKB_CB(skb)->sacked |= TCPCB_REPAIRED; } /* Try to append data to the end of skb. */ if (copy > msg_data_left(msg)) copy = msg_data_left(msg); /* Where to copy to? */ if (skb_availroom(skb) > 0) { /* We have some space in skb head. Superb! */ copy = min_t(int, copy, skb_availroom(skb)); err = skb_add_data_nocache(sk, skb, &msg->msg_iter, copy); if (err) goto do_fault; } else { bool merge = true; int i = skb_shinfo(skb)->nr_frags; struct page_frag *pfrag = sk_page_frag(sk); if (!sk_page_frag_refill(sk, pfrag)) goto wait_for_memory; if (!skb_can_coalesce(skb, i, pfrag->page, pfrag->offset)) { if (i >= sysctl_max_skb_frags || !sg) { tcp_mark_push(tp, skb); goto new_segment; } merge = false; } copy = min_t(int, copy, pfrag->size - pfrag->offset); if (!sk_wmem_schedule(sk, copy)) goto wait_for_memory; err = skb_copy_to_page_nocache(sk, &msg->msg_iter, skb, pfrag->page, pfrag->offset, copy); if (err) goto do_error; /* Update the skb. */ if (merge) { skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); } else { skb_fill_page_desc(skb, i, pfrag->page, pfrag->offset, copy); page_ref_inc(pfrag->page); } pfrag->offset += copy; } if (!copied) TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH; tp->write_seq += copy; TCP_SKB_CB(skb)->end_seq += copy; tcp_skb_pcount_set(skb, 0); copied += copy; if (!msg_data_left(msg)) { if (unlikely(flags & MSG_EOR)) TCP_SKB_CB(skb)->eor = 1; goto out; } if (skb->len < max || (flags & MSG_OOB) || unlikely(tp->repair)) continue; if (forced_push(tp)) { tcp_mark_push(tp, skb); __tcp_push_pending_frames(sk, mss_now, TCP_NAGLE_PUSH); } else if (skb == tcp_send_head(sk)) tcp_push_one(sk, mss_now); continue; wait_for_sndbuf: set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); wait_for_memory: if (copied) tcp_push(sk, flags & ~MSG_MORE, mss_now, TCP_NAGLE_PUSH, size_goal); err = sk_stream_wait_memory(sk, &timeo); if (err != 0) goto do_error; mss_now = tcp_send_mss(sk, &size_goal, flags); } out: if (copied) { tcp_tx_timestamp(sk, sockc.tsflags, tcp_write_queue_tail(sk)); tcp_push(sk, flags, mss_now, tp->nonagle, size_goal); } out_nopush: release_sock(sk); return copied + copied_syn; do_fault: if (!skb->len) { tcp_unlink_write_queue(skb, sk); /* It is the one place in all of TCP, except connection * reset, where we can be unlinking the send_head. */ tcp_check_send_head(sk, skb); sk_wmem_free_skb(sk, skb); } do_error: if (copied + copied_syn) goto out; out_err: err = sk_stream_error(sk, flags, err); /* make sure we wake any epoll edge trigger waiter */ if (unlikely(skb_queue_len(&sk->sk_write_queue) == 0 && err == -EAGAIN)) { sk->sk_write_space(sk); tcp_chrono_stop(sk, TCP_CHRONO_SNDBUF_LIMITED); } release_sock(sk); return err; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds28424.17%11.01%
Linus Torvalds (pre-git)25521.70%2929.29%
Eric Dumazet14712.51%1919.19%
Soheil Hassas Yeganeh716.04%33.03%
Pavel Emelyanov615.19%33.03%
Yuchung Cheng574.85%11.01%
Herbert Xu504.26%22.02%
David S. Miller423.57%77.07%
Jason Baron282.38%11.01%
Al Viro262.21%22.02%
Martin KaFai Lau242.04%11.01%
Arnaldo Carvalho de Melo181.53%1010.10%
Andrey Vagin151.28%11.01%
Alexey Kuznetsov151.28%33.03%
Krishna Kumar121.02%11.01%
Tom Herbert110.94%22.02%
Jerry Chu110.94%11.01%
Wei Wang100.85%11.01%
Francis Yan90.77%11.01%
Ilpo Järvinen80.68%11.01%
Yuvaraja Mariappan80.68%11.01%
Christoph Paasch30.26%11.01%
Changli Gao30.26%22.02%
Hideo Aoki20.17%11.01%
Willem de Bruijn20.17%11.01%
Hans Westgaard Ry10.09%11.01%
Patrick McHardy10.09%11.01%
Stephen Hemminger10.09%11.01%
Total1175100.00%99100.00%

EXPORT_SYMBOL(tcp_sendmsg); /* * Handle reading urgent data. BSD has very simple semantics for * this, no blocking and very strange errors 8) */
static int tcp_recv_urg(struct sock *sk, struct msghdr *msg, int len, int flags) { struct tcp_sock *tp = tcp_sk(sk); /* No URG data to read. */ if (sock_flag(sk, SOCK_URGINLINE) || !tp->urg_data || tp->urg_data == TCP_URG_READ) return -EINVAL; /* Yes this is right ! */ if (sk->sk_state == TCP_CLOSE && !sock_flag(sk, SOCK_DONE)) return -ENOTCONN; if (tp->urg_data & TCP_URG_VALID) { int err = 0; char c = tp->urg_data; if (!(flags & MSG_PEEK)) tp->urg_data = TCP_URG_READ; /* Read urgent data. */ msg->msg_flags |= MSG_OOB; if (len > 0) { if (!(flags & MSG_TRUNC)) err = memcpy_to_msg(msg, &c, 1); len = 1; } else msg->msg_flags |= MSG_TRUNC; return err ? -EFAULT : len; } if (sk->sk_state == TCP_CLOSE || (sk->sk_shutdown & RCV_SHUTDOWN)) return 0; /* Fixed the recv(..., MSG_OOB) behaviour. BSD docs and * the available implementations agree in this case: * this call should never block, independent of the * blocking state of the socket. * Mike <pall@rz.uni-karlsruhe.de> */ return -EAGAIN; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)17489.69%1770.83%
Arnaldo Carvalho de Melo105.15%312.50%
James Morris42.06%14.17%
David S. Miller31.55%14.17%
Linus Torvalds21.03%14.17%
Al Viro10.52%14.17%
Total194100.00%24100.00%


static int tcp_peek_sndq(struct sock *sk, struct msghdr *msg, int len) { struct sk_buff *skb; int copied = 0, err = 0; /* XXX -- need to support SO_PEEK_OFF */ skb_queue_walk(&sk->sk_write_queue, skb) { err = skb_copy_datagram_msg(skb, 0, msg, skb->len); if (err) break; copied += skb->len; } return err ?: copied; }

Contributors

PersonTokensPropCommitsCommitProp
Pavel Emelyanov7398.65%150.00%
David S. Miller11.35%150.00%
Total74100.00%2100.00%

/* Clean up the receive buffer for full frames taken by the user, * then send an ACK if necessary. COPIED is the number of bytes * tcp_recvmsg has given to the user so far, it speeds up the * calculation of whether or not we must ACK for the sake of * a window update. */
static void tcp_cleanup_rbuf(struct sock *sk, int copied) { struct tcp_sock *tp = tcp_sk(sk); bool time_to_ack = false; struct sk_buff *skb = skb_peek(&sk->sk_receive_queue); WARN(skb && !before(tp->copied_seq, TCP_SKB_CB(skb)->end_seq), "cleanup rbuf bug: copied %X seq %X rcvnxt %X\n", tp->copied_seq, TCP_SKB_CB(skb)->end_seq, tp->rcv_nxt); if (inet_csk_ack_scheduled(sk)) { const struct inet_connection_sock *icsk = inet_csk(sk); /* Delayed ACKs frequently hit locked sockets during bulk * receive. */ if (icsk->icsk_ack.blocked || /* Once-per-two-segments ACK was not sent by tcp_input.c */ tp->rcv_nxt - tp->rcv_wup > icsk->icsk_ack.rcv_mss || /* * If this read emptied read buffer, we send ACK, if * connection is not bidirectional, user drained * receive buffer and there was a small segment * in queue. */ (copied > 0 && ((icsk->icsk_ack.pending & ICSK_ACK_PUSHED2) || ((icsk->icsk_ack.pending & ICSK_ACK_PUSHED) && !icsk->icsk_ack.pingpong)) && !atomic_read(&sk->sk_rmem_alloc))) time_to_ack = true; } /* We send an ACK if we can now advertise a non-zero window * which has been raised "significantly". * * Even if window raised up to infinity, do not send window open ACK * in states, where we will not receive more. It is useless. */ if (copied > 0 && !time_to_ack && !(sk->sk_shutdown & RCV_SHUTDOWN)) { __u32 rcv_window_now = tcp_receive_window(tp); /* Optimize, __tcp_select_window() is not cheap. */ if (2*rcv_window_now <= tp->window_clamp) { __u32 new_window = __tcp_select_window(sk); /* Send ACK now, if this read freed lots of space * in our buffer. Certainly, new_window is new window. * We can advertise it now, if it is not less than current one. * "Lots" means "at least twice" here. */ if (new_window && new_window >= 2 * rcv_window_now) time_to_ack = true; } } if (time_to_ack) tcp_send_ack(sk); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)15863.71%1453.85%
Arnaldo Carvalho de Melo3112.50%415.38%
Ilpo Järvinen208.06%27.69%
Linus Torvalds166.45%13.85%
Alexey Kuznetsov145.65%13.85%
Eric Dumazet41.61%13.85%
David S. Miller31.21%13.85%
Christopher Leech10.40%13.85%
Dan J Williams10.40%13.85%
Total248100.00%26100.00%


static void tcp_prequeue_process(struct sock *sk) { struct sk_buff *skb; struct tcp_sock *tp = tcp_sk(sk); NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPPREQUEUED); while ((skb = __skb_dequeue(&tp->ucopy.prequeue)) != NULL) sk_backlog_rcv(sk, skb); /* Clear memory counter. */ tp->ucopy.memory = 0; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)5680.00%222.22%
Pavel Emelyanov57.14%111.11%
David S. Miller34.29%111.11%
Ravikiran G. Thirumalai22.86%111.11%
Arnaldo Carvalho de Melo22.86%222.22%
Hideaki Yoshifuji / 吉藤英明11.43%111.11%
Eric Dumazet11.43%111.11%
Total70100.00%9100.00%


static struct sk_buff *tcp_recv_skb(struct sock *sk, u32 seq, u32 *off) { struct sk_buff *skb; u32 offset; while ((skb = skb_peek(&sk->sk_receive_queue)) != NULL) { offset = seq - TCP_SKB_CB(skb)->seq; if (unlikely(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_SYN)) { pr_err_once("%s: found a SYN, please report !\n", __func__); offset--; } if (offset < skb->len || (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN)) { *off = offset; return skb; } /* This looks weird, but this can happen if TCP collapsing * splitted a fat GRO packet, while we released socket lock * in skb_splice_bits() */ sk_eat_skb(sk, skb); } return NULL; }

Contributors

PersonTokensPropCommitsCommitProp
Trond Myklebust7560.98%120.00%
Eric Dumazet4435.77%360.00%
Arnaldo Carvalho de Melo43.25%120.00%
Total123100.00%5100.00%

/* * This routine provides an alternative to tcp_recvmsg() for routines * that would like to handle copying from skbuffs directly in 'sendfile' * fashion. * Note: * - It is assumed that the socket was locked by the caller. * - The routine does not block. * - At present, there is no support for reading OOB data * or for 'peeking' the socket using this routine * (although both would be easy to implement). */
int tcp_read_sock(struct sock *sk, read_descriptor_t *desc, sk_read_actor_t recv_actor) { struct sk_buff *skb; struct tcp_sock *tp = tcp_sk(sk); u32 seq = tp->copied_seq; u32 offset; int copied = 0; if (sk->sk_state == TCP_LISTEN) return -ENOTCONN; while ((skb = tcp_recv_skb(sk, seq, &offset)) != NULL) { if (offset < skb->len) { int used; size_t len; len = skb->len - offset; /* Stop reading if we hit a patch of urgent data */ if (tp->urg_data) { u32 urg_offset = tp->urg_seq - seq; if (urg_offset < len) len = urg_offset; if (!len) break; } used = recv_actor(desc, skb, offset, len); if (used <= 0) { if (!copied) copied = used; break; } else if (used <= len) { seq += used; copied += used; offset += used; } /* If recv_actor drops the lock (e.g. TCP splice * receive) the skb pointer might be invalid when * getting here: tcp_collapse might have deleted it * while aggregating skbs from the socket queue. */ skb = tcp_recv_skb(sk, seq - 1, &offset); if (!skb) break; /* TCP coalescing might have appended data to the skb. * Try to splice more frags */ if (offset + 1 != skb->len) continue; } if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN) { sk_eat_skb(sk, skb); ++seq; break; } sk_eat_skb(sk, skb); if (!desc->count) break; tp->copied_seq = seq; } tp->copied_seq = seq; tcp_rcv_space_adjust(sk); /* Clean up data we have read: This will do ACK frames. */ if (copied > 0) { tcp_recv_skb(sk, seq, &offset); tcp_cleanup_rbuf(sk, copied); } return copied; }

Contributors

PersonTokensPropCommitsCommitProp
Trond Myklebust22472.96%212.50%
Octavian Purdila216.84%212.50%
Jens Axboe206.51%16.25%
Eric Dumazet175.54%318.75%
Willy Tarreau72.28%16.25%
Steven J. Magnani61.95%16.25%
Arnaldo Carvalho de Melo61.95%425.00%
David S. Miller51.63%16.25%
Christopher Leech10.33%16.25%
Total307100.00%16100.00%

EXPORT_SYMBOL(tcp_read_sock);
int tcp_peek_len(struct socket *sock) { return tcp_inq(sock->sk); }

Contributors

PersonTokensPropCommitsCommitProp
Tom Herbert18100.00%1100.00%
Total18100.00%1100.00%

EXPORT_SYMBOL(tcp_peek_len); /* * This routine copies from a sock struct into the user buffer. * * Technical note: in 2.3 we work on _locked_ socket, so that * tricks with *seq access order and skb->users are not required. * Probably, code can be easily improved even more. */
int tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int nonblock, int flags, int *addr_len) { struct tcp_sock *tp = tcp_sk(sk); int copied = 0; u32 peek_seq; u32 *seq; unsigned long used; int err; int target; /* Read at least this many bytes */ long timeo; struct task_struct *user_recv = NULL; struct sk_buff *skb, *last; u32 urg_hole = 0; if (unlikely(flags & MSG_ERRQUEUE)) return inet_recv_error(sk, msg, len, addr_len); if (sk_can_busy_loop(sk) && skb_queue_empty(&sk->sk_receive_queue) && (sk->sk_state == TCP_ESTABLISHED)) sk_busy_loop(sk, nonblock); lock_sock(sk); err = -ENOTCONN; if (sk->sk_state == TCP_LISTEN) goto out; timeo = sock_rcvtimeo(sk, nonblock); /* Urgent data needs to be handled specially. */ if (flags & MSG_OOB) goto recv_urg; if (unlikely(tp->repair)) { err = -EPERM; if (!(flags & MSG_PEEK)) goto out; if (tp->repair_queue == TCP_SEND_QUEUE) goto recv_sndq; err = -EINVAL; if (tp->repair_queue == TCP_NO_QUEUE) goto out; /* 'common' recv queue MSG_PEEK-ing */ } seq = &tp->copied_seq; if (flags & MSG_PEEK) { peek_seq = tp->copied_seq; seq = &peek_seq; } target = sock_rcvlowat(sk, flags & MSG_WAITALL, len); do { u32 offset; /* Are we at urgent data? Stop if we have read anything or have SIGURG pending. */ if (tp->urg_data && tp->urg_seq == *seq) { if (copied) break; if (signal_pending(current)) { copied = timeo ? sock_intr_errno(timeo) : -EAGAIN; break; } } /* Next get a buffer. */ last = skb_peek_tail(&sk->sk_receive_queue); skb_queue_walk(&sk->sk_receive_queue, skb) { last = skb; /* Now that we have two receive queues this * shouldn't happen. */ if (WARN(before(*seq, TCP_SKB_CB(skb)->seq), "recvmsg bug: copied %X seq %X rcvnxt %X fl %X\n", *seq, TCP_SKB_CB(skb)->seq, tp->rcv_nxt, flags)) break; offset = *seq - TCP_SKB_CB(skb)->seq; if (unlikely(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_SYN)) { pr_err_once("%s: found a SYN, please report !\n", __func__); offset--; } if (offset < skb->len) goto found_ok_skb; if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN) goto found_fin_ok; WARN(!(flags & MSG_PEEK), "recvmsg bug 2: copied %X seq %X rcvnxt %X fl %X\n", *seq, TCP_SKB_CB(skb)->seq, tp->rcv_nxt, flags); } /* Well, if we have backlog, try to process it now yet. */ if (copied >= target && !sk->sk_backlog.tail) break; if (copied) { if (sk->sk_err || sk->sk_state == TCP_CLOSE || (sk->sk_shutdown & RCV_SHUTDOWN) || !timeo || signal_pending(current)) break; } else { if (sock_flag(sk, SOCK_DONE)) break; if (sk->sk_err) { copied = sock_error(sk); break; } if (sk->sk_shutdown & RCV_SHUTDOWN) break; if (sk->sk_state == TCP_CLOSE) { if (!sock_flag(sk, SOCK_DONE)) { /* This occurs when user tries to read * from never connected socket. */ copied = -ENOTCONN; break; } break; } if (!timeo) { copied = -EAGAIN; break; } if (signal_pending(current)) { copied = sock_intr_errno(timeo); break; } } tcp_cleanup_rbuf(sk, copied); if (!sysctl_tcp_low_latency && tp->ucopy.task == user_recv) { /* Install new reader */ if (!user_recv && !(flags & (MSG_TRUNC | MSG_PEEK))) { user_recv = current; tp->ucopy.task = user_recv; tp->ucopy.msg = msg; } tp->ucopy.len = len; WARN_ON(tp->copied_seq != tp->rcv_nxt && !(flags & (MSG_PEEK | MSG_TRUNC))); /* Ugly... If prequeue is not empty, we have to * process it before releasing socket, otherwise * order will be broken at second iteration. * More elegant solution is required!!! * * Look: we have the following (pseudo)queues: * * 1. packets in flight * 2. backlog * 3. prequeue * 4. receive_queue * * Each queue can be processed only if the next ones * are empty. At this point we have empty receive_queue. * But prequeue _can_ be not empty after 2nd iteration, * when we jumped to start of loop because backlog * processing added something to receive_queue. * We cannot release_sock(), because backlog contains * packets arrived _after_ prequeued ones. * * Shortly, algorithm is clear --- to process all * the queues in order. We could make it more directly, * requeueing packets from backlog to prequeue, if * is not empty. It is more elegant, but eats cycles, * unfortunately. */ if (!skb_queue_empty(&tp->ucopy.prequeue)) goto do_prequeue; /* __ Set realtime policy in scheduler __ */ } if (copied >= target) { /* Do not sleep, just process backlog. */ release_sock(sk); lock_sock(sk); } else { sk_wait_data(sk, &timeo, last); } if (user_recv) { int chunk; /* __ Restore normal policy in scheduler __ */ chunk = len - tp->ucopy.len; if (chunk != 0) { NET_ADD_STATS(sock_net(sk), LINUX_MIB_TCPDIRECTCOPYFROMBACKLOG, chunk); len -= chunk; copied += chunk; } if (tp->rcv_nxt == tp->copied_seq && !skb_queue_empty(&tp->ucopy.prequeue)) { do_prequeue: tcp_prequeue_process(sk); chunk = len - tp->ucopy.len; if (chunk != 0) { NET_ADD_STATS(sock_net(sk), LINUX_MIB_TCPDIRECTCOPYFROMPREQUEUE, chunk); len -= chunk; copied += chunk; } } } if ((flags & MSG_PEEK) && (peek_seq - copied - urg_hole != tp->copied_seq)) { net_dbg_ratelimited("TCP(%s:%d): Application bug, race in MSG_PEEK\n", current->comm, task_pid_nr(current)); peek_seq = tp->copied_seq; } continue; found_ok_skb: /* Ok so how much can we use? */ used = skb->len - offset; if (len < used) used = len; /* Do we have urgent data here? */ if (tp->urg_data) { u32 urg_offset = tp->urg_seq - *seq; if (urg_offset < used) { if (!urg_offset) { if (!sock_flag(sk, SOCK_URGINLINE)) { ++*seq; urg_hole++; offset++; used--; if (!used) goto skip_copy; } } else used = urg_offset; } } if (!(flags & MSG_TRUNC)) { err = skb_copy_datagram_msg(skb, offset, msg, used); if (err) { /* Exception. Bailout! */ if (!copied) copied = -EFAULT; break; } } *seq += used; copied += used; len -= used; tcp_rcv_space_adjust(sk); skip_copy: if (tp->urg_data && after(tp->copied_seq, tp->urg_seq)) { tp->urg_data = 0; tcp_fast_path_check(sk); } if (used + offset < skb->len) continue; if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN) goto found_fin_ok; if (!(flags & MSG_PEEK)) sk_eat_skb(sk, skb); continue; found_fin_ok: /* Process the FIN. */ ++*seq; if (!(flags & MSG_PEEK)) sk_eat_skb(sk, skb); break; } while (len > 0); if (user_recv) { if (!skb_queue_empty(&tp->ucopy.prequeue)) { int chunk; tp->ucopy.len = copied > 0 ? len : 0; tcp_prequeue_process(sk); if (copied > 0 && (chunk = len - tp->ucopy.len) != 0) { NET_ADD_STATS(sock_net(sk), LINUX_MIB_TCPDIRECTCOPYFROMPREQUEUE, chunk); len -= chunk; copied += chunk; } } tp->ucopy.task = NULL; tp->ucopy.len = 0; } /* According to UNIX98, msg_name/msg_namelen are ignored * on connected socket. I was just happy when found this 8) --ANK */ /* Clean up data we have read: This will do ACK frames. */ tcp_cleanup_rbuf(sk, copied); release_sock(sk); return copied; out: release_sock(sk); return err; recv_urg: err = tcp_recv_urg(sk, msg, len, flags); goto out; recv_sndq: err = tcp_peek_sndq(sk, msg, len); goto out; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)89366.69%3544.87%
Pavel Emelyanov906.72%33.85%
David S. Miller725.38%810.26%
Ilpo Järvinen372.76%33.85%
Linus Torvalds342.54%11.28%
Arnaldo Carvalho de Melo322.39%78.97%
Alexey Kuznetsov312.32%11.28%
Eliezer Tamir302.24%22.56%
Eric Dumazet272.02%33.85%
Sabrina Dubroca211.57%11.28%
Willem de Bruijn211.57%22.56%
Herbert Xu100.75%11.28%
Ravikiran G. Thirumalai90.67%11.28%
Yuvaraja Mariappan80.60%11.28%
Christopher Leech70.52%22.56%
James Morris60.45%11.28%
Joe Perches40.30%22.56%
Hideaki Yoshifuji / 吉藤英明30.22%11.28%
Arjan van de Ven20.15%11.28%
Stephen Hemminger10.07%11.28%
Al Viro10.07%11.28%
Total1339100.00%78100.00%

EXPORT_SYMBOL(tcp_recvmsg);
void tcp_set_state(struct sock *sk, int state) { int oldstate = sk->sk_state; switch (state) { case TCP_ESTABLISHED: if (oldstate != TCP_ESTABLISHED) TCP_INC_STATS(sock_net(sk), TCP_MIB_CURRESTAB); break; case TCP_CLOSE: if (oldstate == TCP_CLOSE_WAIT || oldstate == TCP_ESTABLISHED) TCP_INC_STATS(sock_net(sk), TCP_MIB_ESTABRESETS); sk->sk_prot->unhash(sk); if (inet_csk(sk)->icsk_bind_hash && !(sk->sk_userlocks & SOCK_BINDPORT_LOCK)) inet_put_port(sk); /* fall through */ default: if (oldstate == TCP_ESTABLISHED) TCP_DEC_STATS(sock_net(sk), TCP_MIB_CURRESTAB); } /* Change state AFTER socket is unhashed to avoid closed * socket sitting in hash tables. */ sk_state_store(sk, state); #ifdef STATE_TRACE SOCK_DEBUG(sk, "TCP sk=%p, State %s -> %s\n", sk, statename[oldstate], statename[state]); #endif }

Contributors

PersonTokensPropCommitsCommitProp
Ilpo Järvinen13287.42%125.00%
Pavel Emelyanov159.93%250.00%
Eric Dumazet42.65%125.00%
Total151100.00%4100.00%

EXPORT_SYMBOL_GPL(tcp_set_state); /* * State processing on a close. This implements the state shift for * sending our FIN frame. Note that we only send a FIN for some * states. A shutdown() may have already sent the FIN, or we may be * closed. */ static const unsigned char new_state[16] = { /* current state: new state: action: */ [0 /* (Invalid) */] = TCP_CLOSE, [TCP_ESTABLISHED] = TCP_FIN_WAIT1 | TCP_ACTION_FIN, [TCP_SYN_SENT] = TCP_CLOSE, [TCP_SYN_RECV] = TCP_FIN_WAIT1 | TCP_ACTION_FIN, [TCP_FIN_WAIT1] = TCP_FIN_WAIT1, [TCP_FIN_WAIT2] = TCP_FIN_WAIT2, [TCP_TIME_WAIT] = TCP_CLOSE, [TCP_CLOSE] = TCP_CLOSE, [TCP_CLOSE_WAIT] = TCP_LAST_ACK | TCP_ACTION_FIN, [TCP_LAST_ACK] = TCP_LAST_ACK, [TCP_LISTEN] = TCP_CLOSE, [TCP_CLOSING] = TCP_CLOSING, [TCP_NEW_SYN_RECV] = TCP_CLOSE, /* should not happen ! */ };
static int tcp_close_state(struct sock *sk) { int next = (int)new_state[sk->sk_state]; int ns = next & TCP_STATE_MASK; tcp_set_state(sk, ns); return next & TCP_ACTION_FIN; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)4297.67%480.00%
Arnaldo Carvalho de Melo12.33%120.00%
Total43100.00%5100.00%

/* * Shutdown the sending side of a connection. Much like close except * that we don't receive shut down or sock_set_flag(sk, SOCK_DEAD). */
void tcp_shutdown(struct sock *sk, int how) { /* We need to grab some memory, and put together a FIN, * and then put it into the queue to be sent. * Tim MacKenzie(tym@dibbler.cs.monash.edu.au) 4 Dec '92. */ if (!(how & SEND_SHUTDOWN)) return; /* If we've already sent a FIN, or it's a closed state, skip this. */ if ((1 << sk->sk_state) & (TCPF_ESTABLISHED | TCPF_SYN_SENT | TCPF_SYN_RECV | TCPF_CLOSE_WAIT)) { /* Clear out any half completed packets. FIN if needed. */ if (tcp_close_state(sk)) tcp_send_fin(sk); } }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)5998.33%685.71%
Arnaldo Carvalho de Melo11.67%114.29%
Total60100.00%7100.00%

EXPORT_SYMBOL(tcp_shutdown);
bool tcp_check_oom(struct sock *sk, int shift) { bool too_many_orphans, out_of_socket_memory; too_many_orphans = tcp_too_many_orphans(sk, shift); out_of_socket_memory = tcp_out_of_memory(sk); if (too_many_orphans) net_info_ratelimited("too many orphaned sockets\n"); if (out_of_socket_memory) net_info_ratelimited("out of memory -- consider tuning tcp_mem\n"); return too_many_orphans || out_of_socket_memory; }

Contributors

PersonTokensPropCommitsCommitProp
Arun Sharma5392.98%133.33%
Joe Perches47.02%266.67%
Total57100.00%3100.00%


void tcp_close(struct sock *sk, long timeout) { struct sk_buff *skb; int data_was_unread = 0; int state; lock_sock(sk); sk->sk_shutdown = SHUTDOWN_MASK; if (sk->sk_state == TCP_LISTEN) { tcp_set_state(sk, TCP_CLOSE); /* Special case. */ inet_csk_listen_stop(sk); goto adjudge_to_death; } /* We need to flush the recv. buffs. We do this only on the * descriptor close, not protocol-sourced closes, because the * reader process may not have drained the data yet! */ while ((skb = __skb_dequeue(&sk->sk_receive_queue)) != NULL) { u32 len = TCP_SKB_CB(skb)->end_seq - TCP_SKB_CB(skb)->seq; if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN) len--; data_was_unread += len; __kfree_skb(skb); } sk_mem_reclaim(sk); /* If socket has been already reset (e.g. in tcp_reset()) - kill it. */ if (sk->sk_state == TCP_CLOSE) goto adjudge_to_death; /* As outlined in RFC 2525, section 2.17, we send a RST here because * data was lost. To witness the awful effects of the old behavior of * always doing a FIN, run an older 2.1.x kernel or 2.0.x, start a bulk * GET in an FTP client, suspend the process, wait for the client to * advertise a zero window, then kill -9 the FTP client, wheee... * Note: timeout is always zero in such a case. */ if (unlikely(tcp_sk(sk)->repair)) { sk->sk_prot->disconnect(sk, 0); } else if (data_was_unread) { /* Unread data was tossed, zap the connection. */ NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONCLOSE); tcp_set_state(sk, TCP_CLOSE); tcp_send_active_reset(sk, sk->sk_allocation); } else if (sock_flag(sk, SOCK_LINGER) && !sk->sk_lingertime) { /* Check zero linger _after_ checking for unread data. */ sk->sk_prot->disconnect(sk, 0); NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONDATA); } else if (tcp_close_state(sk)) { /* We FIN if the application ate all the data before * zapping the connection. */ /* RED-PEN. Formally speaking, we have broken TCP state * machine. State transitions: * * TCP_ESTABLISHED -> TCP_FIN_WAIT1 * TCP_SYN_RECV -> TCP_FIN_WAIT1 (forget it, it's impossible) * TCP_CLOSE_WAIT -> TCP_LAST_ACK * * are legal only when FIN has been sent (i.e. in window), * rather than queued out of window. Purists blame. * * F.e. "RFC state" is ESTABLISHED, * if Linux state is FIN-WAIT-1, but FIN is still not sent. * * The visible declinations are that sometimes * we enter time-wait state, when it is not required really * (harmless), do not send active resets, when they are * required by specs (TCP_ESTABLISHED, TCP_CLOSE_WAIT, when * they look as CLOSING or LAST_ACK for Linux) * Probably, I missed some more holelets. * --ANK * XXX (TFO) - To start off we don't support SYN+ACK+FIN * in a single packet! (May consider it later but will * probably need API support or TCP_CORK SYN-ACK until * data is written and socket is closed.) */ tcp_send_fin(sk); } sk_stream_wait_close(sk, timeout); adjudge_to_death: state = sk->sk_state; sock_hold(sk); sock_orphan(sk); /* It is the last release_sock in its life. It will remove backlog. */ release_sock(sk); /* Now socket is owned by kernel and we acquire BH lock to finish close. No need to check for user refs. */ local_bh_disable(); bh_lock_sock(sk); WARN_ON(sock_owned_by_user(sk)); percpu_counter_inc(sk->sk_prot->orphan_count); /* Have we already been destroyed by a softirq or backlog? */ if (state != TCP_CLOSE && sk->sk_state == TCP_CLOSE) goto out; /* This is a (useful) BSD violating of the RFC. There is a * problem with TCP as specified in that the other end could * keep a socket open forever with no application left this end. * We use a 1 minute timeout (about the same as BSD) then kill * our end. If they send after that then tough - BUT: long enough * that we won't make the old 4*rto = almost no time - whoops * reset mistake. * * Nope, it was not mistake. It is really desired behaviour * f.e. on http servers, when such sockets are useless, but * consume significant resources. Let's do it with special * linger2 option. --ANK */ if (sk->sk_state == TCP_FIN_WAIT2) { struct tcp_sock *tp = tcp_sk(sk); if (tp->linger2 < 0) { tcp_set_state(sk, TCP_CLOSE); tcp_send_active_reset(sk, GFP_ATOMIC); __NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONLINGER); } else { const int tmo = tcp_fin_time(sk); if (tmo > TCP_TIMEWAIT_LEN) { inet_csk_reset_keepalive_timer(sk, tmo - TCP_TIMEWAIT_LEN); } else { tcp_time_wait(sk, TCP_FIN_WAIT2, tmo); goto out; } } } if (sk->sk_state != TCP_CLOSE) { sk_mem_reclaim(sk); if (tcp_check_oom(sk, 0)) { tcp_set_state(sk, TCP_CLOSE); tcp_send_active_reset(sk, GFP_ATOMIC); __NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONMEMORY); } } if (sk->sk_state == TCP_CLOSE) { struct request_sock *req = tcp_sk(sk)->fastopen_rsk; /* We could get here with a non-NULL req if the socket is * aborted (e.g., closed with unread data) before 3WHS * finishes. */ if (req) reqsk_fastopen_remove(sk, req, false); inet_csk_destroy_sock(sk); } /* Otherwise, socket is reprieved until protocol close. */ out: bh_unlock_sock(sk); local_bh_enable(); sock_put(sk); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)33963.48%1430.43%
Pavel Emelyanov499.18%48.70%
Herbert Xu438.05%24.35%
Jerry Chu295.43%12.17%
Arnaldo Carvalho de Melo213.93%817.39%
Eric Dumazet152.81%36.52%
Konstantin Khorenko122.25%12.17%
David S. Miller71.31%36.52%
Hideaki Yoshifuji / 吉藤英明40.75%12.17%
Fengguang Wu30.56%12.17%
Benjamin LaHaise30.56%12.17%
Hideo Aoki20.37%12.17%
James Morris20.37%12.17%
Gerrit Renker10.19%12.17%
Ilpo Järvinen10.19%12.17%
Jesper Juhl10.19%12.17%
Andrew Morton10.19%12.17%
Arun Sharma10.19%12.17%
Total534100.00%46100.00%

EXPORT_SYMBOL(tcp_close); /* These states need RST on ABORT according to RFC793 */
static inline bool tcp_need_reset(int state) { return (1 << state) & (TCPF_ESTABLISHED | TCPF_CLOSE_WAIT | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2 | TCPF_SYN_RECV); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)2689.66%133.33%
David S. Miller26.90%133.33%
Eric Dumazet13.45%133.33%
Total29100.00%3100.00%


int tcp_disconnect(struct sock *sk, int flags) { struct inet_sock *inet = inet_sk(sk); struct inet_connection_sock *icsk = inet_csk(sk); struct tcp_sock *tp = tcp_sk(sk); int err = 0; int old_state = sk->sk_state; if (old_state != TCP_CLOSE) tcp_set_state(sk, TCP_CLOSE); /* ABORT function of RFC793 */ if (old_state == TCP_LISTEN) { inet_csk_listen_stop(sk); } else if (unlikely(tp->repair)) { sk->sk_err = ECONNABORTED; } else if (tcp_need_reset(old_state) || (tp->snd_nxt != tp->write_seq && (1 << old_state) & (TCPF_CLOSING | TCPF_LAST_ACK))) { /* The last check adjusts for discrepancy of Linux wrt. RFC * states */ tcp_send_active_reset(sk, gfp_any()); sk->sk_err = ECONNRESET; } else if (old_state == TCP_SYN_SENT) sk->sk_err = ECONNRESET; tcp_clear_xmit_timers(sk); __skb_queue_purge(&sk->sk_receive_queue); tcp_write_queue_purge(sk); skb_rbtree_purge(&tp->out_of_order_queue); inet->inet_dport = 0; if (!(sk->sk_userlocks & SOCK_BINDADDR_LOCK)) inet_reset_saddr(sk); sk->sk_shutdown = 0; sock_reset_flag(sk, SOCK_DONE); tp->srtt_us = 0; tp->write_seq += tp->max_window + 2; if (tp->write_seq == 0) tp->write_seq = 1; icsk->icsk_backoff = 0; tp->snd_cwnd = 2; icsk->icsk_probes_out = 0; tp->packets_out = 0; tp->snd_ssthresh = TCP_INFINITE_SSTHRESH; tp->snd_cwnd_cnt = 0; tp->window_clamp = 0; tcp_set_ca_state(sk, TCP_CA_Open); tcp_clear_retrans(tp); inet_csk_delack_init(sk); tcp_init_send_head(sk); memset(&tp->rx_opt, 0, sizeof(tp->rx_opt)); __sk_dst_reset(sk); tcp_saved_syn_free(tp); /* Clean up fastopen related fields */ tcp_free_fastopen_req(tp); inet->defer_connect = 0; WARN_ON(inet->inet_num && !icsk->icsk_bind_hash); sk->sk_error_report(sk); return err; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)24565.16%1330.95%
Arnaldo Carvalho de Melo4010.64%1023.81%
David S. Miller256.65%49.52%
Pavel Emelyanov184.79%12.38%
Wei Wang123.19%12.38%
Eric Dumazet123.19%49.52%
Srinivas Aji71.86%12.38%
Yuvaraja Mariappan61.60%12.38%
Ilpo Järvinen41.06%24.76%
Stephen Hemminger30.80%24.76%
James Morris20.53%12.38%
Herbert Xu10.27%12.38%
Yaogong Wang10.27%12.38%
Total376100.00%42100.00%

EXPORT_SYMBOL(tcp_disconnect);
static inline bool tcp_can_repair_sock(const struct sock *sk) { return ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN) && (sk->sk_state != TCP_LISTEN); }

Contributors

PersonTokensPropCommitsCommitProp
Pavel Emelyanov2367.65%125.00%
Eric W. Biedermann720.59%125.00%
Eric Dumazet25.88%125.00%
Andrey Vagin25.88%125.00%
Total34100.00%4100.00%


static int tcp_repair_set_window(struct tcp_sock *tp, char __user *optbuf, int len) { struct tcp_repair_window opt; if (!tp->repair) return -EPERM; if (len != sizeof(opt)) return -EINVAL; if (copy_from_user(&opt, optbuf, sizeof(opt))) return -EFAULT; if (opt.max_window < opt.snd_wnd) return -EINVAL; if (after(opt.snd_wl1, tp->rcv_nxt + opt.rcv_wnd)) return -EINVAL; if (after(opt.rcv_wup, tp->rcv_nxt)) return -EINVAL; tp->snd_wl1 = opt.snd_wl1; tp->snd_wnd = opt.snd_wnd; tp->max_window = opt.max_window; tp->rcv_wnd = opt.rcv_wnd; tp->rcv_wup = opt.rcv_wup; return 0; }

Contributors

PersonTokensPropCommitsCommitProp
Andrey Vagin161100.00%1100.00%
Total161100.00%1100.00%


static int tcp_repair_options_est(struct tcp_sock *tp, struct tcp_repair_opt __user *optbuf, unsigned int len) { struct tcp_repair_opt opt; while (len >= sizeof(opt)) { if (copy_from_user(&opt, optbuf, sizeof(opt))) return -EFAULT; optbuf++; len -= sizeof(opt); switch (opt.opt_code) { case TCPOPT_MSS: tp->rx_opt.mss_clamp = opt.opt_val; break; case TCPOPT_WINDOW: { u16 snd_wscale = opt.opt_val & 0xFFFF; u16 rcv_wscale = opt.opt_val >> 16; if (snd_wscale > 14 || rcv_wscale > 14) return -EFBIG; tp->rx_opt.snd_wscale = snd_wscale; tp->rx_opt.rcv_wscale = rcv_wscale; tp->rx_opt.wscale_ok = 1; } break; case TCPOPT_SACK_PERM: if (opt.opt_val != 0) return -EINVAL; tp->rx_opt.sack_ok |= TCP_SACK_SEEN; if (sysctl_tcp_fack) tcp_enable_fack(tp); break; case TCPOPT_TIMESTAMP: if (opt.opt_val != 0) return -EINVAL; tp->rx_opt.tstamp_ok = 1; break; } } return 0; }

Contributors

PersonTokensPropCommitsCommitProp
Pavel Emelyanov17282.30%266.67%
Andrey Vagin3717.70%133.33%
Total209100.00%3100.00%

/* * Socket option code for TCP. */
static int do_tcp_setsockopt(struct sock *sk, int level, int optname, char __user *optval, unsigned int optlen) { struct tcp_sock *tp = tcp_sk(sk); struct inet_connection_sock *icsk = inet_csk(sk); struct net *net = sock_net(sk); int val; int err = 0; /* These are data/string values, all the others are ints */ switch (optname) { case TCP_CONGESTION: { char name[TCP_CA_NAME_MAX]; if (optlen < 1) return -EINVAL; val = strncpy_from_user(name, optval, min_t(long, TCP_CA_NAME_MAX-1, optlen)); if (val < 0) return -EFAULT; name[val] = 0; lock_sock(sk); err = tcp_set_congestion_control(sk, name); release_sock(sk); return err; } default: /* fallthru */ break; } if (optlen < sizeof(int)) return -EINVAL; if (get_user(val, (int __user *)optval)) return -EFAULT; lock_sock(sk); switch (optname) { case TCP_MAXSEG: /* Values greater than interface MTU won't take effect. However * at the point when this call is done we typically don't yet * know which interface is going to be used */ if (val < TCP_MIN_MSS || val > MAX_TCP_WINDOW) { err = -EINVAL; break; } tp->rx_opt.user_mss = val; break; case TCP_NODELAY: if (val) { /* TCP_NODELAY is weaker than TCP_CORK, so that * this option on corked socket is remembered, but * it is not activated until cork is cleared. * * However, when TCP_NODELAY is set we make * an explicit push, which overrides even TCP_CORK * for currently queued segments. */ tp->nonagle |= TCP_NAGLE_OFF|TCP_NAGLE_PUSH; tcp_push_pending_frames(sk); } else { tp->nonagle &= ~TCP_NAGLE_OFF; } break; case TCP_THIN_LINEAR_TIMEOUTS: if (val < 0 || val > 1) err = -EINVAL; else tp->thin_lto = val; break; case TCP_THIN_DUPACK: if (val < 0 || val > 1) err = -EINVAL; break; case TCP_REPAIR: if (!tcp_can_repair_sock(sk)) err = -EPERM; else if (val == 1) { tp->repair = 1; sk->sk_reuse = SK_FORCE_REUSE; tp->repair_queue = TCP_NO_QUEUE; } else if (val == 0) { tp->repair = 0; sk->sk_reuse = SK_NO_REUSE; tcp_send_window_probe(sk); } else err = -EINVAL; break; case TCP_REPAIR_QUEUE: if (!tp->repair) err = -EPERM; else if (val < TCP_QUEUES_NR) tp->repair_queue = val; else err = -EINVAL; break; case TCP_QUEUE_SEQ: if (sk->sk_state != TCP_CLOSE) err = -EPERM; else if (tp->repair_queue == TCP_SEND_QUEUE) tp->write_seq = val; else if (tp->repair_queue == TCP_RECV_QUEUE) tp->rcv_nxt = val; else err = -EINVAL; break; case TCP_REPAIR_OPTIONS: if (!tp->repair) err = -EINVAL; else if (sk->sk_state == TCP_ESTABLISHED) err = tcp_repair_options_est(tp, (struct tcp_repair_opt __user *)optval, optlen); else err = -EPERM; break; case TCP_CORK: /* When set indicates to always queue non-full frames. * Later the user clears this option and we transmit * any pending partial frames in the queue. This is * meant to be used alongside sendfile() to get properly * filled frames when the user (for example) must write * out headers with a write() call first and then use * sendfile to send out the data parts. * * TCP_CORK can be set together with TCP_NODELAY and it is * stronger than TCP_NODELAY. */ if (val) { tp->nonagle |= TCP_NAGLE_CORK; } else { tp->nonagle &= ~TCP_NAGLE_CORK; if (tp->nonagle&TCP_NAGLE_OFF) tp->nonagle |= TCP_NAGLE_PUSH; tcp_push_pending_frames(sk); } break; case TCP_KEEPIDLE: if (val < 1 || val > MAX_TCP_KEEPIDLE) err = -EINVAL; else { tp->keepalive_time = val * HZ; if (sock_flag(sk, SOCK_KEEPOPEN) && !((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN))) { u32 elapsed = keepalive_time_elapsed(tp); if (tp->keepalive_time > elapsed) elapsed = tp->keepalive_time - elapsed; else elapsed = 0; inet_csk_reset_keepalive_timer(sk, elapsed); } } break; case TCP_KEEPINTVL: if (val < 1 || val > MAX_TCP_KEEPINTVL) err = -EINVAL; else tp->keepalive_intvl = val * HZ; break; case TCP_KEEPCNT: if (val < 1 || val > MAX_TCP_KEEPCNT) err = -EINVAL; else tp->keepalive_probes = val; break; case TCP_SYNCNT: if (val < 1 || val > MAX_TCP_SYNCNT) err = -EINVAL; else icsk->icsk_syn_retries = val; break; case TCP_SAVE_SYN: if (val < 0 || val > 1) err = -EINVAL; else tp->save_syn = val; break; case TCP_LINGER2: if (val < 0) tp->linger2 = -1; else if (val > net->ipv4.sysctl_tcp_fin_timeout / HZ) tp->linger2 = 0; else tp->linger2 = val * HZ; break; case TCP_DEFER_ACCEPT: /* Translate value in seconds to number of retransmits */ icsk->icsk_accept_queue.rskq_defer_accept = secs_to_retrans(val, TCP_TIMEOUT_INIT / HZ, TCP_RTO_MAX / HZ); break; case TCP_WINDOW_CLAMP: if (!val) { if (sk->sk_state != TCP_CLOSE) { err = -EINVAL; break; } tp->window_clamp = 0; } else tp->window_clamp = val < SOCK_MIN_RCVBUF / 2 ? SOCK_MIN_RCVBUF / 2 : val; break; case TCP_QUICKACK: if (!val) { icsk->icsk_ack.pingpong = 1; } else { icsk->icsk_ack.pingpong = 0; if ((1 << sk->sk_state) & (TCPF_ESTABLISHED | TCPF_CLOSE_WAIT) && inet_csk_ack_scheduled(sk)) { icsk->icsk_ack.pending |= ICSK_ACK_PUSHED; tcp_cleanup_rbuf(sk, 1); if (!(val & 1)) icsk->icsk_ack.pingpong = 1; } } break; #ifdef CONFIG_TCP_MD5SIG case TCP_MD5SIG: /* Read the IP->Key mappings from userspace */ err = tp->af_specific->md5_parse(sk, optval, optlen); break; #endif case TCP_USER_TIMEOUT: /* Cap the max time in ms TCP will retry or probe the window * before giving up and aborting (ETIMEDOUT) a connection. */ if (val < 0) err = -EINVAL; else icsk->icsk_user_timeout = msecs_to_jiffies(val); break; case TCP_FASTOPEN: if (val >= 0 && ((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN))) { tcp_fastopen_init_key_once(true); fastopen_queue_tune(sk, val); } else { err = -EINVAL; } break; case TCP_FASTOPEN_CONNECT: if (val > 1 || val < 0) { err = -EINVAL; } else if (sysctl_tcp_fastopen & TFO_CLIENT_ENABLE) { if (sk->sk_state == TCP_CLOSE) tp->fastopen_connect = val; else err = -EINVAL; } else { err = -EOPNOTSUPP; } break; case TCP_TIMESTAMP: if (!tp->repair) err = -EPERM; else tp->tsoffset = val - tcp_time_stamp; break; case TCP_REPAIR_WINDOW: err = tcp_repair_set_window(tp, optval, optlen); break; case TCP_NOTSENT_LOWAT: tp->notsent_lowat = val; sk->sk_write_space(sk); break; default: err = -ENOPROTOOPT; break; } release_sock(sk); return err; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)33127.65%919.15%
Pavel Emelyanov21217.71%36.38%
Christoph Paasch1018.44%24.26%
Stephen Hemminger766.35%12.13%
Linus Torvalds726.02%12.13%
William Allen Simpson685.68%12.13%
Wei Wang584.85%12.13%
Jerry Chu514.26%24.26%
Eric Dumazet443.68%36.38%
Andrey Vagin403.34%24.26%
Arnaldo Carvalho de Melo332.76%612.77%
Hideaki Yoshifuji / 吉藤英明252.09%12.13%
Alexey Kuznetsov221.84%12.13%
Nikolay Borisov141.17%12.13%
David S. Miller131.09%36.38%
Hangbin Liu121.00%12.13%
Julian Anastasov90.75%12.13%
Flavio Leitner40.33%12.13%
Andrew Morton30.25%12.13%
James Morris20.17%12.13%
Andreas Petlund20.17%12.13%
Dmitry Mishin20.17%12.13%
Christopher Leech10.08%12.13%
Al Viro10.08%12.13%
Yuchung Cheng10.08%12.13%
Total1197100.00%47100.00%


int tcp_setsockopt(struct sock *sk, int level, int optname, char __user *optval, unsigned int optlen) { const struct inet_connection_sock *icsk = inet_csk(sk); if (level != SOL_TCP) return icsk->icsk_af_ops->setsockopt(sk, level, optname, optval, optlen); return do_tcp_setsockopt(sk, level, optname, optval, optlen); }

Contributors

PersonTokensPropCommitsCommitProp
Dmitry Mishin7297.30%133.33%
David S. Miller11.35%133.33%
Eric Dumazet11.35%133.33%
Total74100.00%3100.00%

EXPORT_SYMBOL(tcp_setsockopt); #ifdef CONFIG_COMPAT
int compat_tcp_setsockopt(struct sock *sk, int level, int optname, char __user *optval, unsigned int optlen) { if (level != SOL_TCP) return inet_csk_compat_setsockopt(sk, level, optname, optval, optlen); return do_tcp_setsockopt(sk, level, optname, optval, optlen); }

Contributors

PersonTokensPropCommitsCommitProp
Dmitry Mishin5796.61%133.33%
Arnaldo Carvalho de Melo11.69%133.33%
David S. Miller11.69%133.33%
Total59100.00%3100.00%

EXPORT_SYMBOL(compat_tcp_setsockopt); #endif
static void tcp_get_info_chrono_stats(const struct tcp_sock *tp, struct tcp_info *info) { u64 stats[__TCP_CHRONO_MAX], total = 0; enum tcp_chrono i; for (i = TCP_CHRONO_BUSY; i < __TCP_CHRONO_MAX; ++i) { stats[i] = tp->chrono_stat[i - 1]; if (i == tp->chrono_type) stats[i] += tcp_time_stamp - tp->chrono_start; stats[i] *= USEC_PER_SEC / HZ; total += stats[i]; } info->tcpi_busy_time = total; info->tcpi_rwnd_limited = stats[TCP_CHRONO_RWND_LIMITED]; info->tcpi_sndbuf_limited = stats[TCP_CHRONO_SNDBUF_LIMITED]; }

Contributors

PersonTokensPropCommitsCommitProp
Francis Yan119100.00%1100.00%
Total119100.00%1100.00%

/* Return information about state of tcp endpoint in API format. */
void tcp_get_info(struct sock *sk, struct tcp_info *info) { const struct tcp_sock *tp = tcp_sk(sk); /* iff sk_type == SOCK_STREAM */ const struct inet_connection_sock *icsk = inet_csk(sk); u32 now, intv; u64 rate64; bool slow; u32 rate; memset(info, 0, sizeof(*info)); if (sk->sk_type != SOCK_STREAM) return; info->tcpi_state = sk_state_load(sk); /* Report meaningful fields for all TCP states, including listeners */ rate = READ_ONCE(sk->sk_pacing_rate); rate64 = rate != ~0U ? rate : ~0ULL; info->tcpi_pacing_rate = rate64; rate = READ_ONCE(sk->sk_max_pacing_rate); rate64 = rate != ~0U ? rate : ~0ULL; info->tcpi_max_pacing_rate = rate64; info->tcpi_reordering = tp->reordering; info->tcpi_snd_cwnd = tp->snd_cwnd; if (info->tcpi_state == TCP_LISTEN) { /* listeners aliased fields : * tcpi_unacked -> Number of children ready for accept() * tcpi_sacked -> max backlog */ info->tcpi_unacked = sk->sk_ack_backlog; info->tcpi_sacked = sk->sk_max_ack_backlog; return; } slow = lock_sock_fast(sk); info->tcpi_ca_state = icsk->icsk_ca_state; info->tcpi_retransmits = icsk->icsk_retransmits; info->tcpi_probes = icsk->icsk_probes_out; info->tcpi_backoff = icsk->icsk_backoff; if (tp->rx_opt.tstamp_ok) info->tcpi_options |= TCPI_OPT_TIMESTAMPS; if (tcp_is_sack(tp)) info->tcpi_options |= TCPI_OPT_SACK; if (tp->rx_opt.wscale_ok) { info->tcpi_options |= TCPI_OPT_WSCALE; info->tcpi_snd_wscale = tp->rx_opt.snd_wscale; info->tcpi_rcv_wscale = tp->rx_opt.rcv_wscale; } if (tp->ecn_flags & TCP_ECN_OK) info->tcpi_options |= TCPI_OPT_ECN; if (tp->ecn_flags & TCP_ECN_SEEN) info->tcpi_options |= TCPI_OPT_ECN_SEEN; if (tp->syn_data_acked) info->tcpi_options |= TCPI_OPT_SYN_DATA; info->tcpi_rto = jiffies_to_usecs(icsk->icsk_rto); info->tcpi_ato = jiffies_to_usecs(icsk->icsk_ack.ato); info->tcpi_snd_mss = tp->mss_cache; info->tcpi_rcv_mss = icsk->icsk_ack.rcv_mss; info->tcpi_unacked = tp->packets_out; info->tcpi_sacked = tp->sacked_out; info->tcpi_lost = tp->lost_out; info->tcpi_retrans = tp->retrans_out; info->tcpi_fackets = tp->fackets_out; now = tcp_time_stamp; info->tcpi_last_data_sent = jiffies_to_msecs(now - tp->lsndtime); info->tcpi_last_data_recv = jiffies_to_msecs(now - icsk->icsk_ack.lrcvtime); info->tcpi_last_ack_recv = jiffies_to_msecs(now - tp->rcv_tstamp); info->tcpi_pmtu = icsk->icsk_pmtu_cookie; info->tcpi_rcv_ssthresh = tp->rcv_ssthresh; info->tcpi_rtt = tp->srtt_us >> 3; info->tcpi_rttvar = tp->mdev_us >> 2; info->tcpi_snd_ssthresh = tp->snd_ssthresh; info->tcpi_advmss = tp->advmss; info->tcpi_rcv_rtt = jiffies_to_usecs(tp->rcv_rtt_est.rtt)>>3; info->tcpi_rcv_space = tp->rcvq_space.space; info->tcpi_total_retrans = tp->total_retrans; info->tcpi_bytes_acked = tp->bytes_acked; info->tcpi_bytes_received = tp->bytes_received; info->tcpi_notsent_bytes = max_t(int, 0, tp->write_seq - tp->snd_nxt); tcp_get_info_chrono_stats(tp, info); info->tcpi_segs_out = tp->segs_out; info->tcpi_segs_in = tp->segs_in; info->tcpi_min_rtt = tcp_min_rtt(tp); info->tcpi_data_segs_in = tp->data_segs_in; info->tcpi_data_segs_out = tp->data_segs_out; info->tcpi_delivery_rate_app_limited = tp->rate_app_limited ? 1 : 0; rate = READ_ONCE(tp->rate_delivered); intv = READ_ONCE(tp->rate_interval_us); if (rate && intv) { rate64 = (u64)rate * tp->mss_cache * USEC_PER_SEC; do_div(rate64, intv); info->tcpi_delivery_rate = rate64; } unlock_sock_fast(sk, slow); }

Contributors

PersonTokensPropCommitsCommitProp
Herbert Xu33147.97%13.45%
Eric Dumazet19227.83%1551.72%
Yuchung Cheng7611.01%26.90%
Arnaldo Carvalho de Melo385.51%517.24%
Martin KaFai Lau162.32%13.45%
Marcelo Ricardo Leitner162.32%13.45%
Craig Gallek101.45%13.45%
Francis Yan71.01%13.45%
Ilpo Järvinen30.43%13.45%
David S. Miller10.14%13.45%
Total690100.00%29100.00%

EXPORT_SYMBOL_GPL(tcp_get_info);
struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk) { const struct tcp_sock *tp = tcp_sk(sk); struct sk_buff *stats; struct tcp_info info; stats = alloc_skb(5 * nla_total_size_64bit(sizeof(u64)), GFP_ATOMIC); if (!stats) return NULL; tcp_get_info_chrono_stats(tp, &info); nla_put_u64_64bit(stats, TCP_NLA_BUSY, info.tcpi_busy_time, TCP_NLA_PAD); nla_put_u64_64bit(stats, TCP_NLA_RWND_LIMITED, info.tcpi_rwnd_limited, TCP_NLA_PAD); nla_put_u64_64bit(stats, TCP_NLA_SNDBUF_LIMITED, info.tcpi_sndbuf_limited, TCP_NLA_PAD); nla_put_u64_64bit(stats, TCP_NLA_DATA_SEGS_OUT, tp->data_segs_out, TCP_NLA_PAD); nla_put_u64_64bit(stats, TCP_NLA_TOTAL_RETRANS, tp->total_retrans, TCP_NLA_PAD); return stats; }

Contributors

PersonTokensPropCommitsCommitProp
Francis Yan10779.85%150.00%
Yuchung Cheng2720.15%150.00%
Total134100.00%2100.00%


static int do_tcp_getsockopt(struct sock *sk, int level, int optname, char __user *optval, int __user *optlen) { struct inet_connection_sock *icsk = inet_csk(sk); struct tcp_sock *tp = tcp_sk(sk); struct net *net = sock_net(sk); int val, len; if (get_user(len, optlen)) return -EFAULT; len = min_t(unsigned int, len, sizeof(int)); if (len < 0) return -EINVAL; switch (optname) { case TCP_MAXSEG: val = tp->mss_cache; if (!val && ((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN))) val = tp->rx_opt.user_mss; if (tp->repair) val = tp->rx_opt.mss_clamp; break; case TCP_NODELAY: val = !!(tp->nonagle&TCP_NAGLE_OFF); break; case TCP_CORK: val = !!(tp->nonagle&TCP_NAGLE_CORK); break; case TCP_KEEPIDLE: val = keepalive_time_when(tp) / HZ; break; case TCP_KEEPINTVL: val = keepalive_intvl_when(tp) / HZ; break; case TCP_KEEPCNT: val = keepalive_probes(tp); break; case TCP_SYNCNT: val = icsk->icsk_syn_retries ? : net->ipv4.sysctl_tcp_syn_retries; break; case TCP_LINGER2: val = tp->linger2; if (val >= 0) val = (val ? : net->ipv4.sysctl_tcp_fin_timeout) / HZ; break; case TCP_DEFER_ACCEPT: val = retrans_to_secs(icsk->icsk_accept_queue.rskq_defer_accept, TCP_TIMEOUT_INIT / HZ, TCP_RTO_MAX / HZ); break; case TCP_WINDOW_CLAMP: val = tp->window_clamp; break; case TCP_INFO: { struct tcp_info info; if (get_user(len, optlen)) return -EFAULT; tcp_get_info(sk, &info); len = min_t(unsigned int, len, sizeof(info)); if (put_user(len, optlen)) return -EFAULT; if (copy_to_user(optval, &info, len)) return -EFAULT; return 0; } case TCP_CC_INFO: { const struct tcp_congestion_ops *ca_ops; union tcp_cc_info info; size_t sz = 0; int attr; if (get_user(len, optlen)) return -EFAULT; ca_ops = icsk->icsk_ca_ops; if (ca_ops && ca_ops->get_info) sz = ca_ops->get_info(sk, ~0U, &attr, &info); len = min_t(unsigned int, len, sz); if (put_user(len, optlen)) return -EFAULT; if (copy_to_user(optval, &info, len)) return -EFAULT; return 0; } case TCP_QUICKACK: val = !icsk->icsk_ack.pingpong; break; case TCP_CONGESTION: if (get_user(len, optlen)) return -EFAULT; len = min_t(unsigned int, len, TCP_CA_NAME_MAX); if (put_user(len, optlen)) return -EFAULT; if (copy_to_user(optval, icsk->icsk_ca_ops->name, len)) return -EFAULT; return 0; case TCP_THIN_LINEAR_TIMEOUTS: val = tp->thin_lto; break; case TCP_THIN_DUPACK: val = 0; break; case TCP_REPAIR: val = tp->repair; break; case TCP_REPAIR_QUEUE: if (tp->repair) val = tp->repair_queue; else return -EINVAL; break; case TCP_REPAIR_WINDOW: { struct tcp_repair_window opt; if (get_user(len, optlen)) return -EFAULT; if (len != sizeof(opt)) return -EINVAL; if (!tp->repair) return -EPERM; opt.snd_wl1 = tp->snd_wl1; opt.snd_wnd = tp->snd_wnd; opt.max_window = tp->max_window; opt.rcv_wnd = tp->rcv_wnd; opt.rcv_wup = tp->rcv_wup; if (copy_to_user(optval, &opt, len)) return -EFAULT; return 0; } case TCP_QUEUE_SEQ: if (tp->repair_queue == TCP_SEND_QUEUE) val = tp->write_seq; else if (tp->repair_queue == TCP_RECV_QUEUE) val = tp->rcv_nxt; else return -EINVAL; break; case TCP_USER_TIMEOUT: val = jiffies_to_msecs(icsk->icsk_user_timeout); break; case TCP_FASTOPEN: val = icsk->icsk_accept_queue.fastopenq.max_qlen; break; case TCP_FASTOPEN_CONNECT: val = tp->fastopen_connect; break; case TCP_TIMESTAMP: val = tcp_time_stamp + tp->tsoffset; break; case TCP_NOTSENT_LOWAT: val = tp->notsent_lowat; break; case TCP_SAVE_SYN: val = tp->save_syn; break; case TCP_SAVED_SYN: { if (get_user(len, optlen)) return -EFAULT; lock_sock(sk); if (tp->saved_syn) { if (len < tp->saved_syn[0]) { if (put_user(tp->saved_syn[0], optlen)) { release_sock(sk); return -EFAULT; } release_sock(sk); return -EINVAL; } len = tp->saved_syn[0]; if (put_user(len, optlen)) { release_sock(sk); return -EFAULT; } if (copy_to_user(optval, tp->saved_syn + 1, len)) { release_sock(sk); return -EFAULT; } tcp_saved_syn_free(tp); release_sock(sk); } else { release_sock(sk); len = 0; if (put_user(len, optlen)) return -EFAULT; } return 0; } default: return -ENOPROTOOPT; } if (put_user(len, optlen)) return -EFAULT; if (copy_to_user(optval, &val, len)) return -EFAULT; return 0; }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)30129.14%1223.53%
Eric Dumazet26225.36%59.80%
Andrey Vagin11711.33%23.92%
Pavel Emelyanov838.03%23.92%
Stephen Hemminger545.23%23.92%
Eric B Munson504.84%11.96%
Linus Torvalds323.10%47.84%
Arnaldo Carvalho de Melo252.42%713.73%
Nikolay Borisov181.74%23.92%
Joshua Hunt171.65%11.96%
Kenjiro Nakayama131.26%11.96%
Jerry Chu131.26%11.96%
Wei Wang100.97%11.96%
William Allen Simpson90.87%11.96%
Alexey Kuznetsov80.77%11.96%
Julian Anastasov80.77%11.96%
David S. Miller70.68%35.88%
Dmitry Mishin20.19%11.96%
Al Viro20.19%11.96%
Christoph Paasch10.10%11.96%
Yuchung Cheng10.10%11.96%
Total1033100.00%51100.00%


int tcp_getsockopt(struct sock *sk, int level, int optname, char __user *optval, int __user *optlen) { struct inet_connection_sock *icsk = inet_csk(sk); if (level != SOL_TCP) return icsk->icsk_af_ops->getsockopt(sk, level, optname, optval, optlen); return do_tcp_getsockopt(sk, level, optname, optval, optlen); }

Contributors

PersonTokensPropCommitsCommitProp
Dmitry Mishin74100.00%1100.00%
Total74100.00%1100.00%

EXPORT_SYMBOL(tcp_getsockopt); #ifdef CONFIG_COMPAT
int compat_tcp_getsockopt(struct sock *sk, int level, int optname, char __user *optval, int __user *optlen) { if (level != SOL_TCP) return inet_csk_compat_getsockopt(sk, level, optname, optval, optlen); return do_tcp_getsockopt(sk, level, optname, optval, optlen); }

Contributors

PersonTokensPropCommitsCommitProp
Dmitry Mishin5998.33%150.00%
Arnaldo Carvalho de Melo11.67%150.00%
Total60100.00%2100.00%

EXPORT_SYMBOL(compat_tcp_getsockopt); #endif #ifdef CONFIG_TCP_MD5SIG static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool); static DEFINE_MUTEX(tcp_md5sig_mutex); static bool tcp_md5sig_pool_populated = false;
static void __tcp_alloc_md5sig_pool(void) { struct crypto_ahash *hash; int cpu; hash = crypto_alloc_ahash("md5", 0, CRYPTO_ALG_ASYNC); if (IS_ERR(hash)) return; for_each_possible_cpu(cpu) { void *scratch = per_cpu(tcp_md5sig_pool, cpu).scratch; struct ahash_request *req; if (!scratch) { scratch = kmalloc_node(sizeof(union tcp_md5sum_block) + sizeof(struct tcphdr), GFP_KERNEL, cpu_to_node(cpu)); if (!scratch) return; per_cpu(tcp_md5sig_pool, cpu).scratch = scratch; } if (per_cpu(tcp_md5sig_pool, cpu).md5_req) continue; req = ahash_request_alloc(hash, GFP_KERNEL); if (!req) return; ahash_request_set_callback(req, 0, NULL, NULL); per_cpu(tcp_md5sig_pool, cpu).md5_req = req; } /* before setting tcp_md5sig_pool_populated, we must commit all writes * to memory. See smp_rmb() in tcp_get_md5sig_pool() */ smp_wmb(); tcp_md5sig_pool_populated = true; }

Contributors

PersonTokensPropCommitsCommitProp
Eric Dumazet8250.00%457.14%
Herbert Xu4929.88%114.29%
Hideaki Yoshifuji / 吉藤英明3219.51%114.29%
Linus Torvalds10.61%114.29%
Total164100.00%7100.00%


bool tcp_alloc_md5sig_pool(void) { if (unlikely(!tcp_md5sig_pool_populated)) { mutex_lock(&tcp_md5sig_mutex); if (!tcp_md5sig_pool_populated) __tcp_alloc_md5sig_pool(); mutex_unlock(&tcp_md5sig_mutex); } return tcp_md5sig_pool_populated; }

Contributors

PersonTokensPropCommitsCommitProp
Hideaki Yoshifuji / 吉藤英明1947.50%125.00%
Eric Dumazet1230.00%250.00%
Adam Langley922.50%125.00%
Total40100.00%4100.00%

EXPORT_SYMBOL(tcp_alloc_md5sig_pool); /** * tcp_get_md5sig_pool - get md5sig_pool for this user * * We use percpu structure, so if we succeed, we exit with preemption * and BH disabled, to make sure another thread or softirq handling * wont try to get same context. */
struct tcp_md5sig_pool *tcp_get_md5sig_pool(void) { local_bh_disable(); if (tcp_md5sig_pool_populated) { /* coupled with smp_wmb() in __tcp_alloc_md5sig_pool() */ smp_rmb(); return this_cpu_ptr(&tcp_md5sig_pool); } local_bh_enable(); return NULL; }

Contributors

PersonTokensPropCommitsCommitProp
Eric Dumazet2468.57%266.67%
Adam Langley1131.43%133.33%
Total35100.00%3100.00%

EXPORT_SYMBOL(tcp_get_md5sig_pool);
int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *hp, const struct sk_buff *skb, unsigned int header_len) { struct scatterlist sg; const struct tcphdr *tp = tcp_hdr(skb); struct ahash_request *req = hp->md5_req; unsigned int i; const unsigned int head_data_len = skb_headlen(skb) > header_len ? skb_headlen(skb) - header_len : 0; const struct skb_shared_info *shi = skb_shinfo(skb); struct sk_buff *frag_iter; sg_init_table(&sg, 1); sg_set_buf(&sg, ((u8 *) tp) + header_len, head_data_len); ahash_request_set_crypt(req, &sg, NULL, head_data_len); if (crypto_ahash_update(req)) return 1; for (i = 0; i < shi->nr_frags; ++i) { const struct skb_frag_struct *f = &shi->frags[i]; unsigned int offset = f->page_offset; struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT); sg_set_page(&sg, page, skb_frag_size(f), offset_in_page(offset)); ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f)); if (crypto_ahash_update(req)) return 1; } skb_walk_frags(skb, frag_iter) if (tcp_md5_hash_skb_data(hp, frag_iter, 0)) return 1; return 0; }

Contributors

PersonTokensPropCommitsCommitProp
Adam Langley12348.43%110.00%
Christoph Paasch3614.17%110.00%
Eric Dumazet2710.63%440.00%
Herbert Xu259.84%110.00%
William Allen Simpson218.27%110.00%
Hideaki Yoshifuji / 吉藤英明166.30%110.00%
Ian Campbell62.36%110.00%
Total254100.00%10100.00%

EXPORT_SYMBOL(tcp_md5_hash_skb_data);
int tcp_md5_hash_key(struct tcp_md5sig_pool *hp, const struct tcp_md5sig_key *key) { struct scatterlist sg; sg_init_one(&sg, key->key, key->keylen); ahash_request_set_crypt(hp->md5_req, &sg, NULL, key->keylen); return crypto_ahash_update(hp->md5_req); }

Contributors

PersonTokensPropCommitsCommitProp
Christoph Paasch3051.72%133.33%
William Allen Simpson1627.59%133.33%
Herbert Xu1220.69%133.33%
Total58100.00%3100.00%

EXPORT_SYMBOL(tcp_md5_hash_key); #endif
void tcp_done(struct sock *sk) { struct request_sock *req = tcp_sk(sk)->fastopen_rsk; if (sk->sk_state == TCP_SYN_SENT || sk->sk_state == TCP_SYN_RECV) TCP_INC_STATS(sock_net(sk), TCP_MIB_ATTEMPTFAILS); tcp_set_state(sk, TCP_CLOSE); tcp_clear_xmit_timers(sk); if (req) reqsk_fastopen_remove(sk, req, false); sk->sk_shutdown = SHUTDOWN_MASK; if (!sock_flag(sk, SOCK_DEAD)) sk->sk_state_change(sk); else inet_csk_destroy_sock(sk); }

Contributors

PersonTokensPropCommitsCommitProp
Andi Kleen6969.00%125.00%
Jerry Chu2525.00%125.00%
Pavel Emelyanov55.00%125.00%
Eric Dumazet11.00%125.00%
Total100100.00%4100.00%

EXPORT_SYMBOL_GPL(tcp_done);
int tcp_abort(struct sock *sk, int err) { if (!sk_fullsock(sk)) { if (sk->sk_state == TCP_NEW_SYN_RECV) { struct request_sock *req = inet_reqsk(sk); local_bh_disable(); inet_csk_reqsk_queue_drop_and_put(req->rsk_listener, req); local_bh_enable(); return 0; } return -EOPNOTSUPP; } /* Don't race with userspace socket closes such as tcp_close. */ lock_sock(sk); if (sk->sk_state == TCP_LISTEN) { tcp_set_state(sk, TCP_CLOSE); inet_csk_listen_stop(sk); } /* Don't race with BH socket closes such as inet_csk_listen_stop. */ local_bh_disable(); bh_lock_sock(sk); if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_err = err; /* This barrier is coupled with smp_rmb() in tcp_poll() */ smp_wmb(); sk->sk_error_report(sk); if (tcp_need_reset(sk->sk_state)) tcp_send_active_reset(sk, GFP_ATOMIC); tcp_done(sk); } bh_unlock_sock(sk); local_bh_enable(); release_sock(sk); return 0; }

Contributors

PersonTokensPropCommitsCommitProp
Lorenzo Colitti13077.38%266.67%
Eric Dumazet3822.62%133.33%
Total168100.00%3100.00%

EXPORT_SYMBOL_GPL(tcp_abort); extern struct tcp_congestion_ops tcp_reno; static __initdata unsigned long thash_entries;
static int __init set_thash_entries(char *str) { ssize_t ret; if (!str) return 0; ret = kstrtoul(str, 0, &thash_entries); if (ret) return 0; return 1; }

Contributors

PersonTokensPropCommitsCommitProp
Andrew Morton2965.91%150.00%
Eldad Zack1534.09%150.00%
Total44100.00%2100.00%

__setup("thash_entries=", set_thash_entries);
static void __init tcp_init_mem(void) { unsigned long limit = nr_free_buffer_pages() / 16; limit = max(limit, 128UL); sysctl_tcp_mem[0] = limit / 4 * 3; /* 4.68 % */ sysctl_tcp_mem[1] = limit; /* 6.25 % */ sysctl_tcp_mem[2] = sysctl_tcp_mem[0] * 2; /* 9.37 % */ }

Contributors

PersonTokensPropCommitsCommitProp
Glauber de Oliveira Costa5388.33%125.00%
Eric Dumazet46.67%125.00%
Eric W. Biedermann23.33%125.00%
Fabian Frederick11.67%125.00%
Total60100.00%4100.00%


void __init tcp_init(void) { int max_rshare, max_wshare, cnt; unsigned long limit; unsigned int i; BUILD_BUG_ON(sizeof(struct tcp_skb_cb) > FIELD_SIZEOF(struct sk_buff, cb)); percpu_counter_init(&tcp_sockets_allocated, 0, GFP_KERNEL); percpu_counter_init(&tcp_orphan_count, 0, GFP_KERNEL); inet_hashinfo_init(&tcp_hashinfo); tcp_hashinfo.bind_bucket_cachep = kmem_cache_create("tcp_bind_bucket", sizeof(struct inet_bind_bucket), 0, SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); /* Size and allocate the main established and bind bucket * hash tables. * * The methodology is similar to that of the buffer cache. */ tcp_hashinfo.ehash = alloc_large_system_hash("TCP established", sizeof(struct inet_ehash_bucket), thash_entries, 17, /* one slot per 128 KB of memory */ 0, NULL, &tcp_hashinfo.ehash_mask, 0, thash_entries ? 0 : 512 * 1024); for (i = 0; i <= tcp_hashinfo.ehash_mask; i++) INIT_HLIST_NULLS_HEAD(&tcp_hashinfo.ehash[i].chain, i); if (inet_ehash_locks_alloc(&tcp_hashinfo)) panic("TCP: failed to alloc ehash_locks"); tcp_hashinfo.bhash = alloc_large_system_hash("TCP bind", sizeof(struct inet_bind_hashbucket), tcp_hashinfo.ehash_mask + 1, 17, /* one slot per 128 KB of memory */ 0, &tcp_hashinfo.bhash_size, NULL, 0, 64 * 1024); tcp_hashinfo.bhash_size = 1U << tcp_hashinfo.bhash_size; for (i = 0; i < tcp_hashinfo.bhash_size; i++) { spin_lock_init(&tcp_hashinfo.bhash[i].lock); INIT_HLIST_HEAD(&tcp_hashinfo.bhash[i].chain); } cnt = tcp_hashinfo.ehash_mask + 1; sysctl_tcp_max_orphans = cnt / 2; tcp_init_mem(); /* Set per-socket limits to no more than 1/128 the pressure threshold */ limit = nr_free_buffer_pages() << (PAGE_SHIFT - 7); max_wshare = min(4UL*1024*1024, limit); max_rshare = min(6UL*1024*1024, limit); sysctl_tcp_wmem[0] = SK_MEM_QUANTUM; sysctl_tcp_wmem[1] = 16*1024; sysctl_tcp_wmem[2] = max(64*1024, max_wshare); sysctl_tcp_rmem[0] = SK_MEM_QUANTUM; sysctl_tcp_rmem[1] = 87380; sysctl_tcp_rmem[2] = max(87380, max_rshare); pr_info("Hash tables configured (established %u bind %u)\n", tcp_hashinfo.ehash_mask + 1, tcp_hashinfo.bhash_size); tcp_v4_init(); tcp_metrics_init(); BUG_ON(tcp_register_congestion_control(&tcp_reno) != 0); tcp_tasklet_init(); }

Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)12029.27%818.18%
Eric Dumazet9422.93%1227.27%
Arnaldo Carvalho de Melo5413.17%49.09%
John Heffner4110.00%24.55%
Brent Casavant348.29%12.27%
Haishuang Yan92.20%12.27%
Andrew Morton71.71%12.27%
Jason (Hui) Wang61.46%12.27%
Stephen Hemminger61.46%12.27%
Jean Delvare61.46%12.27%
Florian Westphal51.22%12.27%
Glauber de Oliveira Costa51.22%12.27%
Tejun Heo40.98%12.27%
Dimitri Sivanich40.98%12.27%
Tim Bird40.98%12.27%
David S. Miller30.73%12.27%
Alexey Dobriyan20.49%12.27%
Joe Perches20.49%24.55%
Hideo Aoki20.49%12.27%
William Allen Simpson10.24%12.27%
Eric W. Biedermann10.24%12.27%
Total410100.00%44100.00%


Overall Contributors

PersonTokensPropCommitsCommitProp
Linus Torvalds (pre-git)340624.57%9119.70%
Eric Dumazet163311.78%9620.78%
Linus Torvalds11978.64%81.73%
Pavel Emelyanov9176.62%183.90%
Herbert Xu5413.90%132.81%
Arnaldo Carvalho de Melo4483.23%337.14%
Jens Axboe4413.18%20.43%
Andrey Vagin3782.73%61.30%
Ilpo Järvinen3692.66%122.60%
David S. Miller3552.56%286.06%
Yuchung Cheng3482.51%71.52%
Trond Myklebust3032.19%20.43%
Dmitry Mishin2761.99%10.22%
Francis Yan2511.81%30.65%
Wei Wang1811.31%20.43%
Christoph Paasch1761.27%30.65%
Julian Anastasov1521.10%10.22%
Adam Langley1521.10%10.22%
Jerry Chu1501.08%20.43%
Neal Cardwell1501.08%20.43%
Stephen Hemminger1461.05%71.52%
Lorenzo Colitti1350.97%20.43%
Soheil Hassas Yeganeh1350.97%51.08%
William Allen Simpson1250.90%20.43%
Alexey Kuznetsov1130.82%61.30%
Hideaki Yoshifuji / 吉藤英明1110.80%20.43%
Willem de Bruijn950.69%30.65%
Andi Kleen740.53%10.22%
Jason Baron600.43%20.43%
Glauber de Oliveira Costa580.42%10.22%
Andrew Morton560.40%51.08%
Arun Sharma540.39%10.22%
Eric B Munson500.36%10.22%
Mario Schuknecht470.34%10.22%
Martin KaFai Lau460.33%20.43%
Willy Tarreau440.32%30.65%
John Heffner410.30%20.43%
Nikolay Borisov390.28%30.65%
Tom Herbert380.27%40.87%
Brent Casavant370.27%10.22%
Yuvaraja Mariappan340.25%10.22%
Eliezer Tamir330.24%30.65%
Al Viro320.23%61.30%
Hideo Aoki240.17%10.22%
Sabrina Dubroca230.17%10.22%
Eric W. Biedermann220.16%20.43%
Octavian Purdila210.15%20.43%
Krishna Kumar180.13%20.43%
James Morris180.13%10.22%
Joshua Hunt170.12%10.22%
Joe Perches170.12%40.87%
Changli Gao170.12%30.65%
Marcelo Ricardo Leitner160.12%10.22%
Eldad Zack150.11%10.22%
Tom Marshall140.10%10.22%
Ravikiran G. Thirumalai140.10%10.22%
Kenjiro Nakayama130.09%10.22%
Konstantin Khorenko120.09%10.22%
Hangbin Liu120.09%10.22%
Christopher Leech100.07%20.43%
Craig Gallek100.07%10.22%
Haishuang Yan90.06%10.22%
Florian Westphal90.06%10.22%
Ben Hutchings90.06%10.22%
Motohiro Kosaki70.05%10.22%
Tejun Heo70.05%20.43%
Srinivas Aji70.05%10.22%
Steven J. Magnani60.04%10.22%
Jason (Hui) Wang60.04%10.22%
Ian Campbell60.04%10.22%
Miquel van Smoorenburg60.04%10.22%
Jean Delvare60.04%10.22%
David Majnemer50.04%10.22%
Dimitris Michailidis50.04%10.22%
Dimitri Sivanich40.03%10.22%
Tim Bird40.03%10.22%
Hannes Frederic Sowa40.03%10.22%
Flavio Leitner40.03%10.22%
Arjan van de Ven30.02%20.43%
Yaogong Wang30.02%10.22%
Fengguang Wu30.02%10.22%
Benjamin LaHaise30.02%10.22%
Davide Libenzi20.01%10.22%
Patrick McHardy20.01%10.22%
Adrian Bunk20.01%20.43%
Hans Westgaard Ry20.01%10.22%
Andreas Petlund20.01%10.22%
Alexey Dobriyan20.01%10.22%
Pravin B Shelar20.01%10.22%
Will Newton20.01%10.22%
Alexandra N. Kossovsky10.01%10.22%
Gerrit Renker10.01%10.22%
Ian Morris10.01%10.22%
Jesper Juhl10.01%10.22%
Jiri Olsa10.01%10.22%
Dan J Williams10.01%10.22%
Fabian Frederick10.01%10.22%
Satoru SATOH10.01%10.22%
Brian Haley0.00%00.00%
Total13860100.00%462100.00%
Directory: net/ipv4
Information contained on this website is for historical information purposes only and does not indicate or represent copyright ownership.
Created with cregit.