[dts] [PATCH v5 01/22] framework/packet: support packet sniffer to specify running machine
Tu, Lijuan
lijuan.tu at intel.com
Fri Sep 21 10:27:40 CEST 2018
> -----Original Message-----
> From: Phil Yang (Arm Technology China) [mailto:Phil.Yang at arm.com]
> Sent: Friday, September 21, 2018 3:32 PM
> To: Tu, Lijuan <lijuan.tu at intel.com>; Phil at dpdk.org; dts at dpdk.org
> Cc: nd <nd at arm.com>; Liu, Yong <yong.liu at intel.com>
> Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support packet
> sniffer to specify running machine
>
> tcpdump_sniff_packet and load_tcpdump_sniff_pcap in class Tester will call
> those two functions.
>
> def tcpdump_sniff_packets(self, intf, count=0, timeout=5, filters=[]):
> """
> Wrapper for packet module sniff_packets
> """
> # load functions in packet module
> module = __import__("packet")
> sniff_f = getattr(module, "sniff_packets")
>
> target=[]
> target.append(self.get_username())
> target.append(self.get_ip_address())
> target.append(self.get_password())
> return sniff_f(intf, count, timeout, filters, target)
[Lijuan] Here It calls sniff_packets which is in packet.py, but sniff_packets don't use target[2] which means password. If I missed, could you figure out which code would call target[2]?
>
> Please check test.py in this patch.
>
> Thanks,
> Phil Yang
>
> > -----Original Message-----
> > From: Tu, Lijuan <lijuan.tu at intel.com>
> > Sent: Friday, September 21, 2018 3:19 PM
> > To: Phil Yang (Arm Technology China) <Phil.Yang at arm.com>;
> > Phil at dpdk.org; dts at dpdk.org
> > Cc: nd <nd at arm.com>; Liu, Yong <yong.liu at intel.com>
> > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support packet
> > sniffer to specify running machine
> >
> > I am glad we reach an agreement.
> >
> > Of course, the parameter "target" have password, but I found no any
> > function would call it.
> > Only username and ip are used. The target[2] is the password, and is not
> called.
> > + if target:
> > + tcpdump_help_pipe = subprocess.Popen(["ssh",
> > + "%s@%s" % (target[0], target[1]),
> > + "tcpdump -h"],
> > + stderr=subprocess.PIPE,
> > + stdout=subprocess.PIPE,
> > + shell=False)
> > Anything I missed?
> >
> >
> > > -----Original Message-----
> > > From: Phil Yang (Arm Technology China) [mailto:Phil.Yang at arm.com]
> > > Sent: Friday, September 21, 2018 2:59 PM
> > > To: Tu, Lijuan <lijuan.tu at intel.com>; Phil at dpdk.org; dts at dpdk.org
> > > Cc: nd <nd at arm.com>; Liu, Yong <yong.liu at intel.com>
> > > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support packet
> > > sniffer to specify running machine
> > >
> > > Yes. I agreed with that.
> > >
> > > I think my patch has also followed this rule.
> > > In the packet.py, I added one more parameter "target" for
> > > sniff_packets and load_sniff_pcap to specify the username and IP
> address.
> > > In the test.py, I passed the Tester's passwd and IP address to these
> > > two functions. Thus, it will make sure sniff packet operation done
> > > on the
> > Tester.
> > >
> > > Thanks,
> > > Phil Yang
> > >
> > > > -----Original Message-----
> > > > From: Tu, Lijuan <lijuan.tu at intel.com>
> > > > Sent: Friday, September 21, 2018 11:42 AM
> > > > To: Phil Yang (Arm Technology China) <Phil.Yang at arm.com>;
> > > > Phil at dpdk.org; dts at dpdk.org
> > > > Cc: nd <nd at arm.com>; Liu, Yong <yong.liu at intel.com>
> > > > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support
> > > > packet sniffer to specify running machine
> > > >
> > > > DTS supports either login server by password or utilize authorized
> login.
> > > > For logging by password, we could get some codes like "
> > > > self.session = SSHPexpect(host, username, password, dut_id) " in
> > > > SSHConnection module. and we have ip, username, password
> > > > information in crb.cfg I think we don't want to force user to utilize
> authorized login.
> > > >
> > > > > -----Original Message-----
> > > > > From: Phil Yang (Arm Technology China)
> > > > > [mailto:Phil.Yang at arm.com]
> > > > > Sent: Friday, September 21, 2018 11:22 AM
> > > > > To: Tu, Lijuan <lijuan.tu at intel.com>; Phil at dpdk.org;
> > > > > dts at dpdk.org
> > > > > Cc: nd <nd at arm.com>; Liu, Yong <yong.liu at intel.com>
> > > > > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support
> > > > > packet sniffer to specify running machine
> > > > >
> > > > > Hi Lijuan,
> > > > >
> > > > > It also need to follow the DTS setup steps in
> > > > >
> https://doc.dpdk.org/dts/gsg/sys_reqs.html#authorized-login-session .
> > > > >
> > > > > Since you know where are your DUT and Tester, so you can use
> > > > > tool ssh-copy-id to save local available keys on DUT and Tester.
> > > > >
> > > > > Thanks,
> > > > > Phil Yang
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Tu, Lijuan <lijuan.tu at intel.com>
> > > > > > Sent: Friday, September 21, 2018 11:09 AM
> > > > > > To: Phil at dpdk.org; dts at dpdk.org
> > > > > > Cc: nd <nd at arm.com>; Liu, Yong <yong.liu at intel.com>; Phil Yang
> > > > > > (Arm Technology China) <Phil.Yang at arm.com>
> > > > > > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support
> > > > > > packet sniffer to specify running machine
> > > > > >
> > > > > > One comments, when we ssh to a server, it might request a
> > > > > > password, but we don't handle it.
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: dts [mailto:dts-bounces at dpdk.org] On Behalf Of
> > > > > > > Phil at dpdk.org
> > > > > > > Sent: Tuesday, September 4, 2018 4:26 PM
> > > > > > > To: dts at dpdk.org
> > > > > > > Cc: nd at arm.com; Liu, Yong <yong.liu at intel.com>; Phil Yang
> > > > > > > <phil.yang at arm.com>
> > > > > > > Subject: [dts] [PATCH v5 01/22] framework/packet: support
> > > > > > > packet sniffer to specify running machine
> > > > > > >
> > > > > > > From: Phil Yang <phil.yang at arm.com>
> > > > > > >
> > > > > > > By default, the Tester is supposed to be the server which
> > > > > > > running DTS and the packet sniff methods are running on it.
> > > > > > > However, if DTS was not running on the Tester and the Tester
> > > > > > > is another remote server, so packet sniff methods cannot
> > > > > > > sniff Tester's packets correctly.
> > > > > > >
> > > > > > > This patch adds support for packet sniff methods to specify
> > > > > > > running machine and implements sniff packet methods in class
> tester.
> > > > > > >
> > > > > > > 1. Add parameter to sniff_packet and load_sniff_pcap methods
> > > > > > > to specify the running machine.
> > > > > > > 2. Remove load_sniff_packets method in packet.py.
> > > > > > > 3. Add tcpdump_sniff_packets, load_tcpdump_sniff_pcap and
> > > > > > > load_tcpdump_sniff_packets for class tester with the new
> > > > > > > sniff_packet
> > > > > API.
> > > > > > > 4. Update tester.check_random_pkts method with
> > > > > > > tester.tcpdump_sniff_packets and
> > > > > > > tester.load_tcpdump_sniff_packets to make it execution on
> > > > > > > the
> > > Tester.
> > > > > > >
> > > > > > > Signed-off-by: Phil Yang <phil.yang at arm.com>
> > > > > > > Suggested-by: Marvin Liu <yong.liu at intel.com>
> > > > > > > ---
> > > > > > > framework/packet.py | 86
> > > > > > > +++++++++++++++++++++++++----------------------------
> > > > > > > framework/tester.py | 48 +++++++++++++++++++++++++++---
> > > > > > > 2 files changed, 84 insertions(+), 50 deletions(-)
> > > > > > >
> > > > > > > diff --git a/framework/packet.py b/framework/packet.py index
> > > > > > > 976b82b..1f3f07d 100755
> > > > > > > --- a/framework/packet.py
> > > > > > > +++ b/framework/packet.py
> > > > > > > @@ -812,15 +812,31 @@ def get_filter_cmd(filters=[]):
> > > > > > > return ""
> > > > > > >
> > > > > > >
> > > > > > > -def sniff_packets(intf, count=0, timeout=5, filters=[]):
> > > > > > > +def sniff_packets(intf, count=0, timeout=5, filters=[], target=[]):
> > > > > > > """
> > > > > > > sniff all packets for certain port in certain seconds
> > > > > > > """
> > > > > > > param = ""
> > > > > > > direct_param = r"(\s+)\[ (\S+) in\|out\|inout \]"
> > > > > > > - tcpdump_help = subprocess.check_output("tcpdump -h;
> echo
> > > 0",
> > > > > > > -
> > > > > > > stderr=subprocess.STDOUT,
> > > > > > > - shell=True)
> > > > > > > + remote_terminate = 0
> > > > > > > +
> > > > > > > + # target[] contain the remote machine info for ssh
> connection
> > > > > > > + # target[0]: username
> > > > > > > + # target[1]: ip address
> > > > > > > + # target[2]: pass word
> > > > > > > + if target:
> > > > > > > + tcpdump_help_pipe = subprocess.Popen(["ssh",
> > > > > > > + "%s@%s" % (target[0],
> target[1]),
> > > > > > > + "tcpdump -h"],
> > > > > > > + stderr=subprocess.PIPE,
> > > > > > > + stdout=subprocess.PIPE,
> > > > > > > + shell=False)
> > > > > > [Lijuan] Action will be blocked if SSH need password.
> > > > > > > + tcpdump_help =
> > > > > > > "".join(tuple(tcpdump_help_pipe.communicate()))
> > > > > > > + tcpdump_help_pipe.wait()
> > > > > > > + else:
> > > > > > > + tcpdump_help = subprocess.check_output("tcpdump
> -h;
> > > > > > > + echo
> > > > > 0",
> > > > > > > +
> > > > > stderr=subprocess.STDOUT,
> > > > > > > + shell=True)
> > > > > > > +
> > > > > > > for line in tcpdump_help.split('\n'):
> > > > > > > m = re.match(direct_param, line)
> > > > > > > if m:
> > > > > > > @@ -850,22 +866,29 @@ def sniff_packets(intf, count=0,
> > > > > > > timeout=5,
> > > > > > > filters=[]):
> > > > > > > else:
> > > > > > > cmd = sniff_cmd % options
> > > > > > >
> > > > > > > - args = shlex.split(cmd)
> > > > > > > + if target:
> > > > > > > + pipe = subprocess.Popen(['ssh',
> > > > > > > + '%s@%s' % (target[0], target[1]), cmd],
> > > > > > > + stdin=subprocess.PIPE,
> > > > > > > + shell=False)
> > > > > > > + remote_terminate = 1
> > > > > > > + else:
> > > > > > > + args = shlex.split(cmd)
> > > > > > > + pipe = subprocess.Popen(args)
> > > > > > >
> > > > > > > - pipe = subprocess.Popen(args)
> > > > > > > index = str(time.time())
> > > > > > > - SNIFF_PIDS[index] = (pipe, intf, timeout)
> > > > > > > + SNIFF_PIDS[index] = (pipe, intf, timeout,
> > > > > > > + remote_terminate)
> > > > > > > time.sleep(1)
> > > > > > > return index
> > > > > > >
> > > > > > >
> > > > > > > -def load_sniff_pcap(index=''):
> > > > > > > +def load_sniff_pcap(index='', target=[]):
> > > > > > > """
> > > > > > > Stop sniffer and return pcap file
> > > > > > > """
> > > > > > > child_exit = False
> > > > > > > if index in SNIFF_PIDS.keys():
> > > > > > > - pipe, intf, timeout = SNIFF_PIDS[index]
> > > > > > > + pipe, intf, timeout, remote_terminate =
> > > > > > > + SNIFF_PIDS[index]
> > > > > > > time_elapse = int(time.time() - float(index))
> > > > > > > while time_elapse < timeout:
> > > > > > > if pipe.poll() is not None:
> > > > > > > @@ -876,6 +899,14 @@ def load_sniff_pcap(index=''):
> > > > > > > time_elapse += 1
> > > > > > >
> > > > > > > if not child_exit:
> > > > > > > + if remote_terminate == 1:
> > > > > > > + stop_tcpdump_pipe =
> subprocess.Popen(['ssh',
> > > > > > > + '%s@%s' %
> (target[0],
> > > > > > > target[1]),
> > > > > > > + 'kill -2 $(pidof
> > > tcpdump)'],
> > > > > > > +
> stderr=subprocess.PIPE,
> > > > > > > + shell=False)
> > > > > > > + stop_tcpdump_pipe.wait()
> > > > > > > +
> > > > > > > pipe.send_signal(signal.SIGINT)
> > > > > > > pipe.wait()
> > > > > > >
> > > > > > > @@ -886,42 +917,6 @@ def load_sniff_pcap(index=''):
> > > > > > > return ""
> > > > > > >
> > > > > > >
> > > > > > > -def load_sniff_packets(index=''):
> > > > > > > - """
> > > > > > > - Stop sniffer and return packet objects
> > > > > > > - """
> > > > > > > - pkts = []
> > > > > > > - child_exit = False
> > > > > > > - if index in SNIFF_PIDS.keys():
> > > > > > > - pipe, intf, timeout = SNIFF_PIDS[index]
> > > > > > > - time_elapse = int(time.time() - float(index))
> > > > > > > - while time_elapse < timeout:
> > > > > > > - if pipe.poll() is not None:
> > > > > > > - child_exit = True
> > > > > > > - break
> > > > > > > -
> > > > > > > - time.sleep(1)
> > > > > > > - time_elapse += 1
> > > > > > > -
> > > > > > > - if not child_exit:
> > > > > > > - pipe.send_signal(signal.SIGINT)
> > > > > > > - pipe.wait()
> > > > > > > -
> > > > > > > - # wait pcap file ready
> > > > > > > - time.sleep(1)
> > > > > > > - try:
> > > > > > > - cap_pkts = rdpcap("/tmp/sniff_%s.pcap" % intf)
> > > > > > > - for pkt in cap_pkts:
> > > > > > > - # packet gen should be scapy
> > > > > > > - packet = Packet(tx_port=intf)
> > > > > > > - packet.pktgen.assign_pkt(pkt)
> > > > > > > - pkts.append(packet)
> > > > > > > - except:
> > > > > > > - pass
> > > > > > > -
> > > > > > > - return pkts
> > > > > > > -
> > > > > > > -
> > > > > > > def load_pcapfile(filename=""):
> > > > > > > pkts = []
> > > > > > > try:
> > > > > > > @@ -983,7 +978,6 @@ if __name__ == "__main__":
> > > > > > > inst = sniff_packets("lo", timeout=5)
> > > > > > > pkt = Packet(pkt_type='UDP')
> > > > > > > pkt.send_pkt(tx_port='lo')
> > > > > > > - pkts = load_sniff_packets(inst)
> > > > > > >
> > > > > > > pkt = Packet(pkt_type='UDP', pkt_len=1500,
> > > ran_payload=True)
> > > > > > > pkt.send_pkt(tx_port='lo') diff --git
> > > > > > > a/framework/tester.py b/framework/tester.py index
> > > > > > > a775f68..c5b705d 100755
> > > > > > > --- a/framework/tester.py
> > > > > > > +++ b/framework/tester.py
> > > > > > > @@ -35,6 +35,7 @@ Interface for bulk traffic generators.
> > > > > > >
> > > > > > > import re
> > > > > > > import subprocess
> > > > > > > +import os
> > > > > > > from time import sleep
> > > > > > > from settings import NICS, load_global_setting,
> > > > > > > PERF_SETTING from crb import Crb @@ -560,8 +561,6 @@ class
> Tester(Crb):
> > > > > > > module = __import__("packet")
> > > > > > > pkt_c = getattr(module, "Packet")
> > > > > > > send_f = getattr(module, "send_packets")
> > > > > > > - sniff_f = getattr(module, "sniff_packets")
> > > > > > > - load_f = getattr(module, "load_sniff_packets")
> > > > > > > compare_f = getattr(module, "compare_pktload")
> > > > > > > strip_f = getattr(module, "strip_pktload")
> > > > > > > save_f = getattr(module, "save_packets") @@ -606,7
> > > > > > > +605,7 @@ class Tester(Crb):
> > > > > > >
> > > > > > > # send and sniff packets
> > > > > > > save_f(pkts=pkts, filename="/tmp/%s_tx.pcap" %
> > > txIntf)
> > > > > > > - inst = sniff_f(intf=rxIntf, count=pktnum,
> > > > > timeout=timeout,
> > > > > > > filters=
> > > > > > > + inst = self.tcpdump_sniff_packets(intf=rxIntf,
> > > > > > > + count=pktnum, timeout=timeout, filters=
> > > > > > > [{'layer': 'network', 'config': {'srcport':
> '65535'}},
> > > > > > > {'layer': 'network', 'config': {'dstport':
> > > '65535'}}])
> > > > > > > rx_inst[rxport] = inst @@ -627,7 +626,7 @@
> > > > > > > class
> > > > > > > Tester(Crb):
> > > > > > > # Verify all packets
> > > > > > > prev_id = -1
> > > > > > > for txport, rxport in portList:
> > > > > > > - recv_pkts = load_f(rx_inst[rxport])
> > > > > > > + recv_pkts =
> > > > > > > + self.load_tcpdump_sniff_packets(rx_inst[rxport])
> > > > > > >
> > > > > > > # only report when recevied number not matched
> > > > > > > if len(tx_pkts[txport]) > len(recv_pkts):
> > > > > > > @@ -704,6 +703,47 @@ class Tester(Crb):
> > > > > > > self.proc.kill()
> > > > > > > self.proc = None
> > > > > > >
> > > > > > > + def tcpdump_sniff_packets(self, intf, count=0,
> > > > > > > + timeout=5,
> > > > > filters=[]):
> > > > > > > + """
> > > > > > > + Wrapper for packet module sniff_packets
> > > > > > > + """
> > > > > > > + # load functions in packet module
> > > > > > > + module = __import__("packet")
> > > > > > > + sniff_f = getattr(module, "sniff_packets")
> > > > > > > +
> > > > > > > + target=[]
> > > > > > > + target.append(self.get_username())
> > > > > > > + target.append(self.get_ip_address())
> > > > > > > + target.append(self.get_password())
> > > > > > > + return sniff_f(intf, count, timeout, filters,
> > > > > > > + target)
> > > > > > > +
> > > > > > > + def load_tcpdump_sniff_pcap(self, index=''):
> > > > > > > + """
> > > > > > > + Wrapper for packet module load_sniff_pcap
> > > > > > > + """
> > > > > > > + # load functions in packet module
> > > > > > > + module = __import__("packet")
> > > > > > > + load_pcap_f = getattr(module, "load_sniff_pcap")
> > > > > > > +
> > > > > > > + target=[]
> > > > > > > + target.append(self.get_username())
> > > > > > > + target.append(self.get_ip_address())
> > > > > > > + target.append(self.get_password())
> > > > > > > + pcap = load_pcap_f(index, target)
> > > > > > > + self.session.copy_file_from(pcap)
> > > > > > > +
> > > > > > > + return pcap.split(os.sep)[-1]
> > > > > > > +
> > > > > > > + def load_tcpdump_sniff_packets(self, index=''):
> > > > > > > + """
> > > > > > > + Wrapper for packet module load_pcapfile
> > > > > > > + """
> > > > > > > + # load functions in packet module
> > > > > > > + packet = __import__("packet")
> > > > > > > + file = self.load_tcpdump_sniff_pcap(index)
> > > > > > > +
> > > > > > > + return packet.load_pcapfile(file)
> > > > > > > +
> > > > > > > def kill_all(self, killall=False):
> > > > > > > """
> > > > > > > Kill all scapy process or DPDK application on tester.
> > > > > > > --
> > > > > > > 2.7.4
More information about the dts
mailing list