Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 23 Next »

Opening Matt Fredrickson - asterisk project lead

we want to make sure to thank you by having a lunch that is catered, there is some drinks & breakfasts available.

so lots of ways to prevent your blood suagar from dropping

thanks for E4 stratiges for lunch that helps us a lot

also of note we are ending at 15h, that is atypical due to conflicting obligations like a panel
Eric - when will pictures happen
Answer - we will wait until lunchtime

lunch is in the national ballroom B (accross the way)

We do have a speakerphone available & a wireless network called asteriskdevcon that should be more reliable than the astricon wifi

Password is the same digium18
Also want to thank Torrey for taking notes day and George for setting up streaming for those who cannot participate onsite.   The development process is bigger than any one room

Introductions:

Matt Fredrickson - sangoma
Torrey Searle - Voxbone

Malcom Davenport - digium - product manager - here for free food
Jared Smith - digium
Christophe Durieux - connect with the community
Claude Paltry - jive 
Jroan Vinzens- sipgate - bulding a new infrastructure in ast16 with ari
david duffet - digium - observing
fred posner - lod - interestin in the future of asterisk
Ludovic gasc - allo cloud -belgium - interested in ari
pascal cadotte - wazzo - in quebec in future development esp ari
evan boily - wazzo - hope next astricon will be in canada - interested in futre of asterisk
Igor Goncharovskiy - russia - IQ tech unistim developer
Nir Simovitch - isreal - clodonix - uncarrier - here for fun
Eric Klein - clondonix - sponsoring dangerous demos
Lenz Dimitiri - lowe in switserland
Sean McCord - psycore systems - atlanta
Dan Jenkins - nimble ape UK - here to be quiet
andrew naggy - socal - sangoma
james benstrup - writes bugs for freepbx
brian walters - sangoma - buffalo ny - interested in future of asterisk
Jason Parker - atlanta -sangoma - work on phones for sangoma
Dan Collins - usain - knocksville
Paulo Vicentini - Brazil - Voxbone - help with ast13 rollout
Steve Murphy - next vortex - wyoming - 
Peter Weschke - see where asterisk is going with ari
harrison schults - corona - indianapolis
john harden 
alex goodman - corona
ben ford - digium - working on ast for 1 year
kevin harwell -alabama - digium - getting feedback
alejandro traud - frankfurt germany - unemploied - used ast for 6 years 
George Joseph - digium developer




What ever direction you like to see asterisk is within your power.  
What has happened in this last year:
asterisk 15.x 6 bug fix releases, current is 15.6.1
asterisk 14 8 security releases last version is 14.7.7
asterisk 13.x 6 bug fix releases current 13.23.1


3300 merged code reviews (across all branches)

it is amazing the number of contributions, that is a pretty amazing number
that is also means alot of eyeballs, people helping wiht review is strongly encouraged, there are alot of patches pending review a long time because of lack of reviews


1000 commits on asterisk 16 (close 3 per day) 72 individual contributing
asterisk 16 as been released! it has been in feature freeze already for a while.  if you haven't tested it yet, then well patches welcome (smile)

Top contributor
260 Corey Farrel

Standard release vs LTS

something happened last here that broke convention, that made some people angry

in asterisk land we have 2 categories of releases, one is the standard release and the other is the magic LTS

alot of people feel confort in LTS, we decided in lots of caution due to the lots of changes in ast 15 especially ref multi-stream support 
we decided not to make 15 an tls, which broke lots of conventions. On the other hand the changes were supprisingly stable.  they maintained compatiblity with most of the applicaiton interfaces


Standard release
1 year bug fix, 1 year security fix

LTS 
4 year bug fix, 1 year security fix

additionaly we have a new branch inclusion policy.  new features can be put into an existing branch so long as there are tested and are non-major nor breaking changes


that allowed us to alot of neat things with that especially in branch 13, and it helps other branches because all those tests can get moved to the other branches as well

it has been a few years since the last LTS.  16 will be an lts, will be around until 2022, and security fixes til 2023

What's new in 16?

Webrtc video 
webrtc api 
chan_pjsip performance
misc fun

Improving video resilance

  • sensitive to packet loss
  • a loss of a single frame - strange things happen
  • in ast 15 we took a sledge happer approach, if we detected packet loss we requested a full new video frame, (sending an entire video frame uncompressed, -very wasteful)
  • we knew there were some better tech, but didn't have time to use them
  • one tech is NACK - an RTCP control message
    • rtcp is the protocol used in accompanment of rtp, sends statistics
    • the decided to extend rtcp to do other things to allow other things, including NACK
    • it is a negative acknowledgement
    • so if a packet is lost, you send a nack to inform of the lost packet and sender will retransmit, allowing stream repair


      Remb
  • bandwith estimation
  • informs of sender of bandwith limitaitons (I only detect 1.5mb of video, please limit)
  • instant packet repair problem
  • negotiated in sdp (nack is as well)

 

Enhanced Messaging:
Backround - updated all apis to have multiple audio/video streams
last year it looked like people in boxes ( it was a really big deal)
there is some metadata, local video description, remote video description

how on earth do I figure out that is kevin karwell and write in on his box
we added an api to correlate callerid streams with participant metadata
additionally we thought it would be good to add participant to participant messaging to confbridge
allows a one stop shop solution, or to keep messaging close the the media path
uses in-dialog sip MESSAGE, you can set the content type, but only use UTF-8

George did alot of work on the conference bridge, don't want to take away his thunder

PJSIP performace improvements
good talk ref chan_sip vs pjsip last year, large attendence room was packed.  in alot of cases chan_pjsip performed better, but some cases it did not
however not a considerable difference, but we still wanted to address them

approx 7% in cpu usage worse in reg handling
after some perfomance improvements we now have chan_pjsip perform better than chan_sip, 2k req/s for chan_pjsip vs 300 for chan_sip

Misc fun
build improvements: DragonFly BSD, NetBSD, OpenBSD, Solaris 11, FreeBSD

I'm a linux guy so I always fall into the trap of thinking that Linux is the only *nix out there, but Alexander Traud contributed alot for building for other platforms

IPV6 added for Dundi
Does anybody use Dundi? (not that you shouldn't be useing it but it often gets forgoten about)


pjproject updated to v2.8
ability to bundle build libjansson, some versions of jansson have bugs, so bundling helps
misc gccc bug fixes
enhanced messaging with app_sendtext
new databuffer api added to the internal core api of asterisk, allows to do things like packet retransmisison buffers

early media video support in app_dial


python3 compatibility fixes - test suite is largely written in python, and different supporiting scripts

if you are a python developer we would love to see some help in reviewing those scripts to geting them compatible with python 3
sean bright worked on ephemeral DTLS certificates saves alot of pain of cert generation

support for a new cdr backend called beanstalk, developed by Nir 

started looking into performance bottlenecks in stasis message buss, (it's a pubsub message buss) 

improvements to strictrtp - the way we live in a world that is outside the standards of the RFCs, nat is a reality and part of the challenges are that, in sdp there is an address it expects to recieve media on.  In order to work in that world you have to believe that address.  If the other end is a private ip address that isn't true.  some of the things that are done are symmetricip, where asterisk sends rtp back to the same address it recieved on, this is all find, but it has to trust the rtp stream is the right stream.  But this can lead to hijacking of rtp streams.


Strict rtp is on by default, prevents of rtp hijacking can't happen, it may cause limitations when lots of video streams are happening, or crappy networks that send packets in big clumps, that presented problems in strictrtp, some improvements were added to make that better. 


Reminder ast14 went EOL a couple weeks ago.  Ast15 graduated to security fixes only mode.  Keep track of what's happening in non-lts major releases of asterisk.
keep a test environment with non-lts releases to avoid supprises,

ast17 is presumably a standard relase so good opportunity to start breaking things


Questions:

when shall we see 16?
Answer:

right now! downloads.asterisk.org is up


Agenda for today
Presentation by George Joseph and Kevin Karwell about performance enhancements in 16 (30 min)

presentation of Ben Ford (originally Josh Colp presentation) about new video improvements (30 min)

Luch 12-1

 

Discussions:

  • about proposing that chan_sip be marked deprecated
  • how does sangoma implact the future of digium
  • opportunistic dtls
  • res_monitor deprecation
  • making easier to stream audio in a way to get it out of asterisk (ari) especially in context of speech recognition
  • getting access to end of call statistics (regardless of who hangs up first)
  • getting rid of extensions.conf for ARI applications
  • discuss around asterisk pushing to a message queue
  • discussion around removal of macro functionality
  • improvement to access media streams
  • res_xmpp & chan_motif should they remain core supported?
  • Alexander Traud - RTCP with audio codecs 

 

George Joseph | kevin Harwell presentation presentation:

Asterisk 2018 Performance Update

Matt gave a preview this morning but we are going to go in more detail

all of us would define performance a bit different as we have different usage patters & network environments and introducing their own deifintion of performance

so it is hard to define performance and what it means to improve it.  CPU? CPS? memory? disk io?

where do improvements come from?

every release have performance improvements, some from community, some from core.  even minor releases have perf improvements.  you should always check the changes file as some perf changes might have functional implications too

Key improvements:

 

rmudgett spent a good amount of time with CDRs to reduce the time and resources to process it.  

it doesn't really impact cps or quality, so do we care? yes, it takes reasources away from those other things.

stasis is another area, it's our internal pubsub buss.  over the last 1.5 months we have done alot of work to reduce the overhead. one of the areas we discovered was that app_voicemail when you start it, and you have holding turned on, app_voicemail wants to know which mailboxes currently have subscribers so it only polls the mailboxes with subscribers. 

most people don't have that turned on, collected via stasis, and in order to do that stasis had to keep track of every mailbox, it had to also have to keep track of the message flowing on the buss & never flushed cached.  even if you didn't have polling turned on then you had alot of memory growth, which in turned caused cpu growth.  all for app_voicemail.  We cut that out, resulting in significat memory & cpu reduction

 

qualify & options, alot of attention this last year, one issue especially with realtime db, the time it takes to iterate contacts, to find which ones need to have qualify, and also write it back.  lots of startup overhead.  similarly inbound reistrations are the same way.  Very read/write dependent.  if you have thousands of aors and you are building up this db of contacts it can get really slow.  pjsip show contacts is forced to go back to the database just to show it on the console

kevin will show some results now:

the thing we mainly focused on was cpu utilization, but you will also see some slides about memory

CDRs -30% increase in record processing, thanks richard

stasis cache - before memory climbing, after cpu is stablized and memory too

pjsip contacts

prior to 13.21.0

poor load and reload time, low reg/s

asterisk 13.21.0

improved registrations per second
asterisk 13.22.0 and after

context re-write patch had the biggest effect for improving performance

reloading contacts now take less than a second

 

Inbound registrations

pjsip channel was kind of lacking on registration perfomance (as mentioned last year) we took that seriously

setup: 1. socket 2 core 2 threads per core i3-2330 cpu 2.20GHZ (cpu will be averaged across all cpus)

regs are written to a database 

we only loaded the modules required for the test (only loaded registration module)

4000 endpoints

we used sipp scenario to over local network

res_pjsip uses 7-8% more cpu than chan_sip, not many retransmissions

13.23.0 now uses less cpu than chan_sip for registration traffic (1-2% less cpu usage)

asterisk 16.1.0

cpu has dropped even more (3% less cpu than chan sip) mem use dropped by 3-4 mb

chan_sip can safely handle 300req/s but will choke on retransmisisons after

res_pjsip can easily handle that & much more, here you can see it easily handle 2k reg/s, at 2.5k/s retransmissions did climb a bit, but still ok

in the latest version of asterisk should see more improvements, some will be shipped in 16.1, be sure to test

testing is a huge part, so please let us know what you find (patches are welcome too)

Questions?

which format do you write cdrs in, can you specify the format?

answer: we don't have a json formatter for cdrs yet

answer: we have different backens, e.g. database, adding a json backend should be easy to add

answer: improvements should be across the board regardless of the backend tech

question: what was the motivation of performace evaluation

answer: there was a presentation last year that did a comparison of chan_sip vs chan_pjsip

answer: there should be a blogpost with a link to the tests

question: in the presentaiton ref performace, there was also a performance hit ref concurrent calls

answer: concurrent calls depends on media processing, chan_pjsip and chan_sip share the same underlaying media processing, so we confused by that and have no idea how to test that

answer: the delta between the two was only 1-2 concurrent call difference, so we were looking for more extreme differences that

answer: if anything could make the difference, it could be memory as pjsip does use more memory

answer: don't have any data-points but I found that chan_pjsip can handle more concurrent calls than chan_sip

answer: found a performance difference between 1.4 and 1.6 and it came down to a one line change ref DNS, there should be more performance tests on a regular basis

answer: yes, we are attempting to focus more on performance, we can def use more feedback, especially on what things we should test, indeed DNS can make a huge difference

 

Ben Ford
How not to trust the quality of the internet

Shattering the illusion some have that the traffic on the internet always get from point A to point B

 

today we will talk about media improvements particularly ref video

I'm not josh & don't have his knowlege, send him or me questions bford@digium.com

Overview:

patchy networks cause packet loss, which causes awful video experience

we are going to talk about how REMB & NACK work within asterisk w some demos

 

packet loss causes:

routing options - sub-obtimal routing (it may go via california)

network congestion - some packets won't get there

some packets may end up out of order

 

this hurts communicaiton if you are missing packets then you are obviosuly going to get (more apparent in video)

video freezes, or video stops

audio just gets choppy or "wonky" sounding

webrtc has tech to help video REMB & nack

What is remb?

remote estimated max bitrate

what this does is poll all the users of a call and find the bandwith of each user individually

it applys to all media streams

it can change throughout the lifetime of a call - so things can change during the call

it can control the video encoding bitrate of the sender, telling the sender the range we can expect/allow

supported in webrtc land

 

how does remb work

each receiver going to calculate max bitrate, then send it to asteris

asterisk relays or generates it's own feedback message

 

REMB is just a first step, transportcc and tmmbr, not widely supported though, foundation is in place to add later

 

NACK is like a ACK but not really it's a negative acknowledgement

it is a message when you do not get something you expect, eg report a lost packet

can be used to trigger retransmission & counter the effects of packet loss

works on a per-stream basis

no guarantee to get all lost packets back, (you may loose the packets again)

 

how does nack work?

ingresss -store packets in a buffer, so we can resend if needed

egress - if we recieve a packet we are expecting process it

otherwise store it, any missing packets then send a NACK request

when buffer reaches 1/2 size we send a nack to get the missing packets

 

if we get packets out of order, we store them, it's possible for the missing packet to still come late, if it does we process it, then get any subsequent packets from the buffer

 

conference bridges behave differently for remb than for 1 on 1

we provde a way to combine reports, we do that by adding options

avg max and min bitrate

depending on which reports we receive we send either lower/highets/avg we receive to the other participants

 

it's possible to receive nack request from multiple participants asterisk needs to determine which stream it belongs to

 

 

 

  • No labels