File size: 62,669 Bytes
f043c61
 
b6818aa
f043c61
 
 
 
 
1d9f226
c24e305
 
f043c61
bd1c868
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f043c61
eb70347
f043c61
bd1c868
f043c61
b6818aa
f043c61
bd1c868
6162409
c24e305
b6818aa
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
 
7547048
b6818aa
bd1c868
 
 
 
 
 
 
 
 
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
b6818aa
bd1c868
 
1d9f226
b6818aa
1d9f226
bd1c868
1d9f226
 
b6818aa
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
bd1c868
1d9f226
b6818aa
1d9f226
bd1c868
1d9f226
 
b6818aa
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
1d9f226
b6818aa
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f043c61
6c0b011
f043c61
fed2156
b9c9c50
f043c61
 
 
 
 
fed2156
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6f51558
 
 
 
 
 
 
 
 
 
 
 
fed2156
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b9c9c50
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
---
language: en
license: apache-2.0
tags:
- bart
- seq2seq
- summarization
datasets:
- cnndaily/newyorkdaily/xsum/samsum/dialogsum/AMI
metrics:
- rouge
widget:
- text: 'Hi, I''m David and I''m supposed to be an industrial designer. Um, I just
    got the project announcement about what the project is. Designing a remote control.
    That''s about it, didn''t get anything else. Did you get the same thing? Cool.
    There''s too much gear. Okay. Can''t draw. Um. Yeah. Um, well anyway, I don''t
    know, it''s just the first animal I can think off the top of my head. Um. Yes.
    Big reason is ''cause I''m allergic to most animals. Allergic to animal fur, so
    um fish was a natural choice. Um, yeah, and I kind of like whales. They come in
    and go eat everything in sight. And they''re quite harmless and mild and interesting.
    Tail''s a bit big, I think. It''s an after dinner dog then. Hmm. It does make
    sense from maybe the design point of view ''cause you have more complicated characters
    like European languages, then you need more buttons. So, possibly. Hmm. Yeah.
    And you keep losing them. Finding them is really a pain, you know. I mean it''s
    usually quite small, or when you want it right, it slipped behind the couch or
    it''s kicked under the table. You know. Yep. Mm-hmm. I think one factor would
    be production cost. Because there''s a cap there, so um depends on how much you
    can cram into that price. Um. I think that that''s the main factor. Cool.

    Okay. Right. Um well this is the kick-off meeting for our our project. Um and
    um this is just what we''re gonna be doing over the next twenty five minutes.
    Um so first of all, just to kind of make sure that we all know each other, I''m
    Laura and I''m the project manager. Do you want to introduce yourself again? Okay.
    Great. Okay. Um so we''re designing a new remote control and um Oh I have to record
    who''s here actually. So that''s David, Andrew and Craig, isn''t it? And you all
    arrived on time. Um yeah so des uh design a new remote control. Um, as you can
    see it''s supposed to be original, trendy and user friendly. Um so that''s kind
    of our our brief, as it were. Um and so there are three different stages to the
    design. Um I''m not really sure what what you guys have already received um in
    your emails. What did you get? Mm-hmm. Is that what everybody got? Okay. Um. So
    we''re gonna have like individual work and then a meeting about it. And repeat
    that process three times. Um and at this point we get try out the whiteboard over
    there. Um. So uh you get to draw your favourite animal and sum up your favourite
    characteristics of it. So who would like to go first? Very good. Mm-hmm. Yeah.
    Yeah. Right. Lovely. Right. You can take as long over this as you like, because
    we haven''t got an awful lot to discuss. Ok oh we do we do. Don''t feel like you''re
    in a rush, anyway. Ach why not We might have to get you up again then. I don''t
    know what mine is. I''m gonna have to think on the spot now. Is that a whale?
    Ah. Okay. God, I still don''t know what I''m gonna write about. Um. I was gonna
    choose a dog as well. But I''ll just draw a different kind of dog. M my favourite
    animal is my own dog at home. Um That doesn''t really look like him, actually.
    He looks more like a pig, actually. Ah well. Do you? Oh that''s very good of you.
    Uh. Um he''s a mixture of uh various things. Um and what do I like about him,
    um That''s just to suggest that his tail wags. Um he''s very friendly and cheery
    and always pleased to see you, and very kind of affectionate and um uh and he''s
    quite quite wee as well so you know he can doesn''t take up too much space. Um
    and uh And he does a funny thing where he chases his tail as well, which is quite
    amusing, so It is. I think it is. He only does it after he''s had his dinner and
    um he''ll just all of a sudden just get up and start chasing his tail ''round
    the living room. Yeah, so uh Yeah, maybe. Maybe. Right, um where did you find
    this? Just down here? Yeah. Okay. Um what are we doing next? Uh um. Okay, uh we
    now need to discuss the project finance. Um so according to the brief um we''re
    gonna be selling this remote control for twenty five Euro, um and we''re aiming
    to make fifty million Euro. Um so we''re gonna be selling this on an international
    scale. And uh we don''t want it to cost any more than uh twelve fifty Euros, so
    fifty percent of the selling price. Sure. All together. Um I dunno. I imagine
    That''s a good question. I imagine it probably is our sale actually because it''s
    probably up to the the um the retailer to uh sell it for whatever price they want.
    Um. But I I don''t know, I mean do you think the fact that it''s going to be sold
    internationally will have a bearing on how we design it at all? Think it will?
    Um. Hmm. Oh yeah, regions and stuff, yeah. Yeah. Okay. Yeah. Well for a remote
    control, do you think that will be I suppose it''s depends on how complicated
    our remote control is. Yeah, yeah. Okay. What, just like in terms of like the
    wealth of the country? Like how much money people have to spend on things like?
    Aye, I see what you mean, yeah. Marketing. Good marketing thoughts. Oh gosh, I
    should be writing all this down. Um. Mm. Yeah. Yeah, yeah. Like how much does,
    you know, a remote control cost. Well twenty five Euro, I mean that''s um that''s
    about like eighteen pounds or something, isn''t it? Or no, is it as much as that?
    Sixteen seventeen eighteen pounds. Um, I dunno, I''ve never bought a remote control,
    so I don''t know how how good a remote control that would get you. Um. But yeah,
    I suppose it has to look kind of cool and gimmicky. Um right, okay. Let me just
    scoot on ahead here. Okay. Um well d Does anybody have anything to add to uh to
    the finance issue at all? Thin No, actually. That would be useful, though, wouldn''t
    it, if you knew like what your money would get you now. Mm-hmm. Yeah, yeah. Oh.
    Five minutes to end of meeting. Oh, okay. We''re a bit behind. Yeah. Right, so
    do you think that should be like a main design aim of our remote control d you
    know, do your your satellite and your regular telly and your V_C_R_ and everything?
    Mm-hmm. Yeah. Or even like, you know, notes about um what you wanna watch. Like
    you might put in there oh I want to watch such and such and look a Oh that''s
    a good idea. So extra functionalities. Mm-hmm. Hmm. Um okay, uh I''d wel we''re
    gonna have to wrap up pretty quickly in the next couple of minutes. Um I''ll just
    check we''ve nothing else. Okay. Um so anything else anybody wants to add about
    what they don''t like about remote controls they''ve used, what they would really
    like to be part of this new one at all? You keep losing them. Okay. Yeah. W You
    get those ones where you can, if you like, whistle or make a really high pitched
    noise they beep. There I mean is that something we''d want to include, do you
    think? Dunno. Okay maybe. My goodness. Still feels quite primitive. Maybe like
    a touch screen or something? Okay. Uh-huh, okay. Well I guess that''s up to our
    industrial designer. It looks better. Yeah. Okay. Okay. Right, well um so just
    to wrap up, the next meeting''s gonna be in thirty minutes. So that''s about um
    about ten to twelve by my watch. Um so inbetween now and then, um as the industrial
    designer, you''re gonna be working on you know the actual working design of it
    so y you know what you''re doing there. Um for user interface, technical functions,
    I guess that''s you know like what we''ve been talking about, what it''ll actually
    do. Um and uh marketing executive, you''ll be just thinking about what it actually
    what, you know, what requirements it has to has to fulfil and you''ll all get
    instructions emailed to you, I guess. Um. Yeah, so it''s th the functional design
    stage is next, I guess. And uh and that''s the end of the meeting. So I got that
    little message a lot sooner than I thought I would, so Mm-hmm. Uh-huh, yeah. Th
    Okay, well just very quickly ''cause this we''re supposed to finish now. Um I
    guess that''s up to us, I mean you probably want some kind of unique selling point
    of it, so um, you know Yeah. Mm-hmm. Yeah. Okay. Right, okay, we''ll that''s that''s
    the end of the meeting, then. Um. So, uh thank you all for coming.

    Um I''m Craig and I''m User Interface. Yeah. Well, my favourite animal would be
    a monkey. Then they''re small cute and furry, and uh when planet of the apes becomes
    real, I''m gonna be up there with them. Yeah. I know um My parents went out and
    bought um remote controls because um they got fed up of having four or five different
    remote controls for each things the house. So um for them it was just how many
    devices control. Uh.

    Mm-hmm. Great. And I''m Andrew and I''m uh our marketing expert. Mm-hmm. Mm-hmm.
    Yeah, that''s that''s it. Yeah. I will go. That''s fine. Alright. So This one
    here, right? Okay. Very nice. Alright. My favourite animal is like A beagle. Um
    charac favourite characteristics of it? Is that right? Uh, right, well basically
    um high priority for any animal for me is that they be willing to take a lot of
    physical affection from their family. And, yeah that they have lots of personality
    and uh be fit and in robust good health. So this is blue. Blue beagle. My family''s
    beagle. I coulda told you a whole lot more about beagles. Boy, let me tell you.
    Impressionist. Alright. Mm. Superb sketch, by the way. Yep. I see a dog in there.
    Yep. Now I see a rooster. What kind is it? Is he aware that th it''s his own cha
    tail he''s chasing? Hmm. Probably when he was little he got lots of attention
    for doing it and has forever been conditioned. ''Kay. Um, can we just go over
    that again? Uh, so bas at twel Alright, yeah. Okay. So cost like production cost
    is twelve fifty, but selling price is is that wholesale or retail? Like on the
    shelf. Our sale our sale anyway. Yeah, okay okay. Okay. Mm-hmm. Alright. Yes.
    Mm-hmm. Mm-hmm. Well right away I''m wondering if there''s um th th uh, like with
    D_V_D_ players, if there are zones. Um f frequencies or something um as well as
    uh characters, um different uh keypad styles and s symbols. Um. I don''t know.
    Yeah. Yeah. Yeah. And then a and then al the other thing international is on top
    of the price. I''m thinking the price might might appeal to a certain market in
    one region, whereas in another it''ll be different, so Just a chara just a characteristic
    of the Just Or just like, basic product podi positioning, the twenty five Euro
    remote control might be a big hit in London, might not be such a big hit in Greece,
    who knows, something like that, yeah. Yep. Right away I''m making some kind of
    assumptions about what what information we''re given here, thinking, ''kay trendy
    probably means something other than just basic, something other than just standard.
    Um so I''m wondering right away, is selling twenty five Euros, is that sort of
    the thi is this gonna to be like the premium product kinda thing or Uh-huh. Mm-hmm.
    Yep. Yeah, I''d say so, yeah. No. Yeah, yeah. Mm-hmm. Do we have any other background
    information on like how that compares to other other Yeah. Mm-hmm. Yeah, interesting
    thing about discussing um production of a remote control for me is that l as you
    point out, I just don''t think of remote controls as somethin something people
    consciously assess in their purchasing habits. It''s just like getting shoelaces
    with shoes or something. It just comes along. Do you know what I mean? Like so
    sort of like how do you I I mean one one way of looking at it would be, well the
    people producing television sets, maybe they have to buy remote controls. Or another
    way is maybe people who have T_V_ sets are really fed up with their remote control
    and they really want a better one or something. But Right. Right. Okay so Right,
    so in function one of the priorities might be to combine as many uses I think
    so. Yeah, yeah. Yeah. Well like um, maybe what we could use is a sort of like
    a example of a successful other piece technology is palm palm pilots. They''re
    gone from being just like little sort of scribble boards to cameras, M_P_ three
    players, telephones, everything, agenda. So, like, I wonder if we might add something
    new to the to the remote control market, such as the lighting in your house, or
    um Yeah, yeah. An Yeah. Like, p personally for me, at home I''ve I''ve combined
    the um the audio video of my television set and my D_V_D_ player and my C_D_ player.
    So they w all work actually function together but I have different remote controls
    for each of them. So it''s sort of ironic that that then they''re in there um
    you know, the sound and everything it''s just one system. But each one''s got
    its own little part. Mm. Mm. Mm. Mm-hmm. Mm-hmm. Yeah. Yeah. That''s just really
    good id Yep. Uh, sure. I remember when the first remote control my my family had
    was on a cable. Actually had a cable between it and the T_V_ and big like buttons
    that sort of like, like on a blender or something. And um, you know, when I think
    about what they are now, it''s better, but actually it''s still kind of, I dunno,
    like a massive junky thing on the table. Maybe we could think about how, could
    be more, you know, streamlined. S Something like that, yeah. Or whatever would
    be technologically reasonable. ''Cause it could b it could it could be that f
    it could be that functionally that doesn''t make it any better, but that just
    the appeal of of not having You know, these days there''s a r pe things in people''s
    homes are becoming more and more like chic, you know. Um, nicer materials and
    might be be worth exploring anyway. Okay. Um. Before we wrap up, just to make
    sure we''re all on the same page here, um, do we We were given sort of an example
    of a coffee machine or something, right? Well, um are we at ma right now on the
    assumption that our television remote control may have features which go beyond
    the television? Or are we keeping sort of like a a design commitment to television
    features? I I don''t know. Yep. Yeah, sure. Okay. Okay, yeah. Okay. Okay. Okay.
    Alright.'
model-index:
- name: MEETING_SUMMARY
  results:
  - task:
      type: abstractive-text-summarization
      name: Abstractive Text Summarization
    dataset:
      name: samsum
      type: samsum
    metrics:
    - type: rouge-1
      value: 53.8795
      name: Validation ROGUE-1
    - type: rouge-2
      value: 28.4975
      name: Validation ROGUE-2
    - type: rouge-L
      value: 44.1899
      name: Validation ROGUE-L
    - type: rouge-Lsum
      value: 49.4863
      name: Validation ROGUE-Lsum
    - type: gen-length
      value: 30.088
      name: Validation ROGUE-Lsum
    - type: rouge-1
      value: 53.2284
      name: Test ROGUE-1
    - type: rouge-2
      value: 28.184
      name: Test ROGUE-2
    - type: rouge-L
      value: 44.122
      name: Test ROGUE-L
    - type: rouge-Lsum
      value: 49.0301
      name: Test ROGUE-Lsum
    - type: gen-length
      value: 29.9951
      name: Test ROGUE-Lsum
  - task:
      type: summarization
      name: Summarization
    dataset:
      name: bazzhangz/sumdataset
      type: bazzhangz/sumdataset
      config: bazzhangz--sumdataset
      split: train
    metrics:
    - type: rouge
      value: 40.5544
      name: ROUGE-1
      verified: true
    - type: rouge
      value: 17.0751
      name: ROUGE-2
      verified: true
    - type: rouge
      value: 32.153
      name: ROUGE-L
      verified: true
    - type: rouge
      value: 36.4277
      name: ROUGE-LSUM
      verified: true
    - type: loss
      value: 2.116729736328125
      name: loss
      verified: true
    - type: gen_len
      value: 42.1978
      name: gen_len
      verified: true
  - task:
      type: abstractive-text-summarization
      name: Abstractive Text Summarization
    dataset:
      name: xsum
      type: xsum
    metrics:
    - type: rouge-1
      value: 35.9078
      name: Validation ROGUE-1
    - type: rouge-2
      value: 14.2497
      name: Validation ROGUE-2
    - type: rouge-L
      value: 28.1421
      name: Validation ROGUE-L
    - type: rouge-Lsum
      value: 28.9826
      name: Validation ROGUE-Lsum
    - type: gen-length
      value: 32.0167
      name: Validation ROGUE-Lsum
    - type: rouge-1
      value: 36.0241
      name: Test ROGUE-1
    - type: rouge-2
      value: 14.3715
      name: Test ROGUE-2
    - type: rouge-L
      value: 28.1968
      name: Test ROGUE-L
    - type: rouge-Lsum
      value: 29.0527
      name: Test ROGUE-Lsum
    - type: gen-length
      value: 31.9933
      name: Test ROGUE-Lsum
  - task:
      type: abstractive-text-summarization
      name: Abstractive Text Summarization
    dataset:
      name: dialogsum
      type: dialogsum
    metrics:
    - type: rouge-1
      value: 39.8612
      name: Validation ROGUE-1
    - type: rouge-2
      value: 16.6917
      name: Validation ROGUE-2
    - type: rouge-L
      value: 32.2718
      name: Validation ROGUE-L
    - type: rouge-Lsum
      value: 35.8748
      name: Validation ROGUE-Lsum
    - type: gen-length
      value: 41.726
      name: Validation ROGUE-Lsum
    - type: rouge-1
      value: 36.9608
      name: Test ROGUE-1
    - type: rouge-2
      value: 14.3058
      name: Test ROGUE-2
    - type: rouge-L
      value: 29.3261
      name: Test ROGUE-L
    - type: rouge-Lsum
      value: 32.9
      name: Test ROGUE-Lsum
    - type: gen-length
      value: 43.086
      name: Test ROGUE-Lsum
  - task:
      type: summarization
      name: Summarization
    dataset:
      name: samsum
      type: samsum
      config: samsum
      split: test
    metrics:
    - type: rouge
      value: 53.1878
      name: ROUGE-1
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTVkNTczYjFmYzBmMzczNWE0MGY4MDAyZWExOGNjZmY1Yzk2ZGM1MGNjZmFmYWUyZmIxZjdjOTk4OTc4OGJlMSIsInZlcnNpb24iOjF9.yyzPpGtESuZXy_lBESrboGxdGYB7I6jaIjquCYqliE2xdbGf5awDFpDUwlZHDuw6RD2mIZv1FC8PPs9lOHuSAg
    - type: rouge
      value: 28.1666
      name: ROUGE-2
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjAzOTdjNGYxNWMzYmFjYjRmMTcxYzI0MmNlNmM5Nzg2MzBlNDdmZWFkN2EwMDE2ZTZmYzc0Zjg0ZDc0M2IxNiIsInZlcnNpb24iOjF9.cPH6O50T6HekO227Xzha-EN_Jp7JS9fh5EP9I0tHxbpGptKtZOQC-NG68zfU2eJKlRSrmgaBYs8tjfTvpAgyDg
    - type: rouge
      value: 44.117
      name: ROUGE-L
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNmNmMzJkYjMxMjhlZDM4YmU3NmI1MDExNzhiYmVhMzEyZGJjNDJkNzczNGQwOTMwNzg2YjU1ZWQ4MDhiMzkxYiIsInZlcnNpb24iOjF9.lcEXK15UqZOdXnPjVqIhFd6o_PLROSIONTRFX5NbwanjEI_MWMLpDh_V0Kpnvs_W0sE6cXh2yoifSYNDA5W7Bw
    - type: rouge
      value: 49.0094
      name: ROUGE-LSUM
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYThkYjk4ZjMzYjI0OTAxNDJiZTU5MzE0YjI5MjEzYTYwNWEzMmU5NjU2ZjQ5NzJhMzkyNmVhNWFjZmM1MjAwMSIsInZlcnNpb24iOjF9.LTn6LpKuMO4Rv4NgsbPmtr2ewiKyoqAXlf6YJfM_6GKwVTKpnJxwx7gaaAtMb0jVlgieITMP11JmbeRfMEhgDg
    - type: loss
      value: 1.710614562034607
      name: loss
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjNjZmM0ZjkwYWYyMWIyMmFiMWI1ODBiYjRjNzVhM2JhN2NmNmM1ZDUwZWRjNDQxNzUwMWM4YjYxYTg1MWYwNyIsInZlcnNpb24iOjF9.hGXZhp9pe-HDJilXVvMCkqz-92YZvH6Qr7q9Z7fJkm8N9s0b4sl-4PwjQYJEOLEAhoRO2s-F5T3bmCYCaMiNBQ
    - type: gen_len
      value: 29.9951
      name: gen_len
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmY1NzZiMDAzNGJlNTg4Nzc0YzU1MTA3YTI3MzVmNGZkNWQ0ZDE4MGZlNGI1MzJmYzA3MjQ0MDZhMTcyYTk2NCIsInZlcnNpb24iOjF9.8dvMfY7Y-nw-K8NGgTXIGFMxaSUWQYBE1w3N5YYOn4iwnCe2ugo2qPIOxLY91q7CaAOMCSskFV3BDStQ4p0ZCg
---
Model obtained by Fine Tuning 'facebook/bart-large-xsum' using AMI Meeting Corpus, SAMSUM Dataset, DIALOGSUM Dataset, XSUM Dataset!
## Usage
# Example 1
```python
from transformers import pipeline
summarizer = pipeline("summarization", model="knkarthick/MEETING_SUMMARY")
text = '''The tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building, and the tallest structure in Paris. Its base is square, measuring 125 metres (410 ft) on each side. During its construction, the Eiffel Tower surpassed the Washington Monument to become the tallest man-made structure in the world, a title it held for 41 years until the Chrysler Building in New York City was finished in 1930. It was the first structure to reach a height of 300 metres. Due to the addition of a broadcasting aerial at the top of the tower in 1957, it is now taller than the Chrysler Building by 5.2 metres (17 ft). Excluding transmitters, the Eiffel Tower is the second tallest free-standing structure in France after the Millau Viaduct.                                       
'''
summarizer(text)
```
# Example 2
```python
from transformers import pipeline
summarizer = pipeline("summarization", model="knkarthick/MEETING_SUMMARY")
text = '''Bangalore is the capital and the largest city of the Indian state of Karnataka. It has a population of more than 8 million and a metropolitan population of around 11 million, making it the third most populous city and fifth most populous urban agglomeration in India. Located in southern India on the Deccan Plateau, at a height of over 900 m (3,000 ft) above sea level, Bangalore is known for its pleasant climate throughout the year. Its elevation is the highest among the major cities of India.The city's history dates back to around 890 CE, in a stone inscription found at the Nageshwara Temple in Begur, Bangalore. The Begur inscription is written in Halegannada (ancient Kannada), mentions 'Bengaluru Kalaga' (battle of Bengaluru). It was a significant turning point in the history of Bangalore as it bears the earliest reference to the name 'Bengaluru'. In 1537 CE, Kempé Gowdā – a feudal ruler under the Vijayanagara Empire – established a mud fort considered to be the foundation of modern Bangalore and its oldest areas, or petes, which exist to the present day.
After the fall of Vijayanagar empire in 16th century, the Mughals sold Bangalore to Chikkadevaraja Wodeyar (1673–1704), the then ruler of the Kingdom of Mysore for three lakh rupees. When Haider Ali seized control of the Kingdom of Mysore, the administration of Bangalore passed into his hands. 
The city  was captured by the British East India Company after victory in the Fourth Anglo-Mysore War (1799), who returned administrative control of the city to the Maharaja of Mysore. The old city developed in the dominions of the Maharaja of Mysore and was made capital of the Princely State of Mysore, which existed as a nominally sovereign entity of the British Raj. In 1809, the British shifted their cantonment to Bangalore, outside the old city, and a town grew up around it, which was governed as part of British India. Following India's independence in 1947, Bangalore became the capital of Mysore State, and remained capital when the new Indian state of Karnataka was formed in 1956. The two urban settlements of Bangalore – city and cantonment – which had developed as independent entities merged into a single urban centre in 1949. The existing Kannada name, Bengalūru, was declared the official name of the city in 2006.
Bangalore is widely regarded as the "Silicon Valley of India" (or "IT capital of India") because of its role as the nation's leading information technology (IT) exporter. Indian technological organisations are headquartered in the city. A demographically diverse city, Bangalore is the second fastest-growing major metropolis in India. Recent estimates of the metro economy of its urban area have ranked Bangalore either the fourth- or fifth-most productive metro area of India. As of 2017, Bangalore was home to 7,700 millionaires and 8 billionaires with a total wealth of $320 billion. It is home to many educational and research institutions. Numerous state-owned aerospace and defence organisations are located in the city. The city also houses the Kannada film industry. It was ranked the most liveable Indian city with a population of over a million under the Ease of Living Index 2020.
'''
summarizer(text)
```

# Example 3
```python
from transformers import pipeline
summarizer = pipeline("summarization", model="knkarthick/MEETING_SUMMARY")
text = '''Hi, I'm David and I'm supposed to be an industrial designer. Um, I just got the project announcement about what the project is. Designing a remote control. That's about it, didn't get anything else. Did you get the same thing? Cool. There's too much gear. Okay. Can't draw. Um. Yeah. Um, well anyway, I don't know, it's just the first animal I can think off the top of my head. Um. Yes. Big reason is 'cause I'm allergic to most animals. Allergic to animal fur, so um fish was a natural choice. Um, yeah, and I kind of like whales. They come in and go eat everything in sight. And they're quite harmless and mild and interesting. Tail's a bit big, I think. It's an after dinner dog then. Hmm. It does make sense from maybe the design point of view 'cause you have more complicated characters like European languages, then you need more buttons. So, possibly. Hmm. Yeah. And you keep losing them. Finding them is really a pain, you know. I mean it's usually quite small, or when you want it right, it slipped behind the couch or it's kicked under the table. You know. Yep. Mm-hmm. I think one factor would be production cost. Because there's a cap there, so um depends on how much you can cram into that price. Um. I think that that's the main factor. Cool.
Okay. Right. Um well this is the kick-off meeting for our our project. Um and um this is just what we're gonna be doing over the next twenty five minutes. Um so first of all, just to kind of make sure that we all know each other, I'm Laura and I'm the project manager. Do you want to introduce yourself again? Okay. Great. Okay. Um so we're designing a new remote control and um Oh I have to record who's here actually. So that's David, Andrew and Craig, isn't it? And you all arrived on time. Um yeah so des uh design a new remote control. Um, as you can see it's supposed to be original, trendy and user friendly. Um so that's kind of our our brief, as it were. Um and so there are three different stages to the design. Um I'm not really sure what what you guys have already received um in your emails. What did you get? Mm-hmm. Is that what everybody got? Okay. Um. So we're gonna have like individual work and then a meeting about it. And repeat that process three times. Um and at this point we get try out the whiteboard over there. Um. So uh you get to draw your favourite animal and sum up your favourite characteristics of it. So who would like to go first? Very good. Mm-hmm. Yeah. Yeah. Right. Lovely. Right. You can take as long over this as you like, because we haven't got an awful lot to discuss. Ok oh we do we do. Don't feel like you're in a rush, anyway. Ach why not We might have to get you up again then. I don't know what mine is. I'm gonna have to think on the spot now. Is that a whale? Ah. Okay. God, I still don't know what I'm gonna write about. Um. I was gonna choose a dog as well. But I'll just draw a different kind of dog. M my favourite animal is my own dog at home. Um That doesn't really look like him, actually. He looks more like a pig, actually. Ah well. Do you? Oh that's very good of you. Uh. Um he's a mixture of uh various things. Um and what do I like about him, um That's just to suggest that his tail wags. Um he's very friendly and cheery and always pleased to see you, and very kind of affectionate and um uh and he's quite quite wee as well so you know he can doesn't take up too much space. Um and uh And he does a funny thing where he chases his tail as well, which is quite amusing, so It is. I think it is. He only does it after he's had his dinner and um he'll just all of a sudden just get up and start chasing his tail 'round the living room. Yeah, so uh Yeah, maybe. Maybe. Right, um where did you find this? Just down here? Yeah. Okay. Um what are we doing next? Uh um. Okay, uh we now need to discuss the project finance. Um so according to the brief um we're gonna be selling this remote control for twenty five Euro, um and we're aiming to make fifty million Euro. Um so we're gonna be selling this on an international scale. And uh we don't want it to cost any more than uh twelve fifty Euros, so fifty percent of the selling price. Sure. All together. Um I dunno. I imagine That's a good question. I imagine it probably is our sale actually because it's probably up to the the um the retailer to uh sell it for whatever price they want. Um. But I I don't know, I mean do you think the fact that it's going to be sold internationally will have a bearing on how we design it at all? Think it will? Um. Hmm. Oh yeah, regions and stuff, yeah. Yeah. Okay. Yeah. Well for a remote control, do you think that will be I suppose it's depends on how complicated our remote control is. Yeah, yeah. Okay. What, just like in terms of like the wealth of the country? Like how much money people have to spend on things like? Aye, I see what you mean, yeah. Marketing. Good marketing thoughts. Oh gosh, I should be writing all this down. Um. Mm. Yeah. Yeah, yeah. Like how much does, you know, a remote control cost. Well twenty five Euro, I mean that's um that's about like eighteen pounds or something, isn't it? Or no, is it as much as that? Sixteen seventeen eighteen pounds. Um, I dunno, I've never bought a remote control, so I don't know how how good a remote control that would get you. Um. But yeah, I suppose it has to look kind of cool and gimmicky. Um right, okay. Let me just scoot on ahead here. Okay. Um well d Does anybody have anything to add to uh to the finance issue at all? Thin No, actually. That would be useful, though, wouldn't it, if you knew like what your money would get you now. Mm-hmm. Yeah, yeah. Oh. Five minutes to end of meeting. Oh, okay. We're a bit behind. Yeah. Right, so do you think that should be like a main design aim of our remote control d you know, do your your satellite and your regular telly and your V_C_R_ and everything? Mm-hmm. Yeah. Or even like, you know, notes about um what you wanna watch. Like you might put in there oh I want to watch such and such and look a Oh that's a good idea. So extra functionalities. Mm-hmm. Hmm. Um okay, uh I'd wel we're gonna have to wrap up pretty quickly in the next couple of minutes. Um I'll just check we've nothing else. Okay. Um so anything else anybody wants to add about what they don't like about remote controls they've used, what they would really like to be part of this new one at all? You keep losing them. Okay. Yeah. W You get those ones where you can, if you like, whistle or make a really high pitched noise they beep. There I mean is that something we'd want to include, do you think? Dunno. Okay maybe. My goodness. Still feels quite primitive. Maybe like a touch screen or something? Okay. Uh-huh, okay. Well I guess that's up to our industrial designer. It looks better. Yeah. Okay. Okay. Right, well um so just to wrap up, the next meeting's gonna be in thirty minutes. So that's about um about ten to twelve by my watch. Um so inbetween now and then, um as the industrial designer, you're gonna be working on you know the actual working design of it so y you know what you're doing there. Um for user interface, technical functions, I guess that's you know like what we've been talking about, what it'll actually do. Um and uh marketing executive, you'll be just thinking about what it actually what, you know, what requirements it has to has to fulfil and you'll all get instructions emailed to you, I guess. Um. Yeah, so it's th the functional design stage is next, I guess. And uh and that's the end of the meeting. So I got that little message a lot sooner than I thought I would, so Mm-hmm. Uh-huh, yeah. Th Okay, well just very quickly 'cause this we're supposed to finish now. Um I guess that's up to us, I mean you probably want some kind of unique selling point of it, so um, you know Yeah. Mm-hmm. Yeah. Okay. Right, okay, we'll that's that's the end of the meeting, then. Um. So, uh thank you all for coming.
Um I'm Craig and I'm User Interface. Yeah. Well, my favourite animal would be a monkey. Then they're small cute and furry, and uh when planet of the apes becomes real, I'm gonna be up there with them. Yeah. I know um My parents went out and bought um remote controls because um they got fed up of having four or five different remote controls for each things the house. So um for them it was just how many devices control. Uh.
Mm-hmm. Great. And I'm Andrew and I'm uh our marketing expert. Mm-hmm. Mm-hmm. Yeah, that's that's it. Yeah. I will go. That's fine. Alright. So This one here, right? Okay. Very nice. Alright. My favourite animal is like A beagle. Um charac favourite characteristics of it? Is that right? Uh, right, well basically um high priority for any animal for me is that they be willing to take a lot of physical affection from their family. And, yeah that they have lots of personality and uh be fit and in robust good health. So this is blue. Blue beagle. My family's beagle. I coulda told you a whole lot more about beagles. Boy, let me tell you. Impressionist. Alright. Mm. Superb sketch, by the way. Yep. I see a dog in there. Yep. Now I see a rooster. What kind is it? Is he aware that th it's his own cha tail he's chasing? Hmm. Probably when he was little he got lots of attention for doing it and has forever been conditioned. 'Kay. Um, can we just go over that again? Uh, so bas at twel Alright, yeah. Okay. So cost like production cost is twelve fifty, but selling price is is that wholesale or retail? Like on the shelf. Our sale our sale anyway. Yeah, okay okay. Okay. Mm-hmm. Alright. Yes. Mm-hmm. Mm-hmm. Well right away I'm wondering if there's um th th uh, like with D_V_D_ players, if there are zones. Um f frequencies or something um as well as uh characters, um different uh keypad styles and s symbols. Um. I don't know. Yeah. Yeah. Yeah. And then a and then al the other thing international is on top of the price. I'm thinking the price might might appeal to a certain market in one region, whereas in another it'll be different, so Just a chara just a characteristic of the Just Or just like, basic product podi positioning, the twenty five Euro remote control might be a big hit in London, might not be such a big hit in Greece, who knows, something like that, yeah. Yep. Right away I'm making some kind of assumptions about what what information we're given here, thinking, 'kay trendy probably means something other than just basic, something other than just standard. Um so I'm wondering right away, is selling twenty five Euros, is that sort of the thi is this gonna to be like the premium product kinda thing or Uh-huh. Mm-hmm. Yep. Yeah, I'd say so, yeah. No. Yeah, yeah. Mm-hmm. Do we have any other background information on like how that compares to other other Yeah. Mm-hmm. Yeah, interesting thing about discussing um production of a remote control for me is that l as you point out, I just don't think of remote controls as somethin something people consciously assess in their purchasing habits. It's just like getting shoelaces with shoes or something. It just comes along. Do you know what I mean? Like so sort of like how do you I I mean one one way of looking at it would be, well the people producing television sets, maybe they have to buy remote controls. Or another way is maybe people who have T_V_ sets are really fed up with their remote control and they really want a better one or something. But Right. Right. Okay so Right, so in function one of the priorities might be to combine as many uses I think so. Yeah, yeah. Yeah. Well like um, maybe what we could use is a sort of like a example of a successful other piece technology is palm palm pilots. They're gone from being just like little sort of scribble boards to cameras, M_P_ three players, telephones, everything, agenda. So, like, I wonder if we might add something new to the to the remote control market, such as the lighting in your house, or um Yeah, yeah. An Yeah. Like, p personally for me, at home I've I've combined the um the audio video of my television set and my D_V_D_ player and my C_D_ player. So they w all work actually function together but I have different remote controls for each of them. So it's sort of ironic that that then they're in there um you know, the sound and everything it's just one system. But each one's got its own little part. Mm. Mm. Mm. Mm-hmm. Mm-hmm. Yeah. Yeah. That's just really good id Yep. Uh, sure. I remember when the first remote control my my family had was on a cable. Actually had a cable between it and the T_V_ and big like buttons that sort of like, like on a blender or something. And um, you know, when I think about what they are now, it's better, but actually it's still kind of, I dunno, like a massive junky thing on the table. Maybe we could think about how, could be more, you know, streamlined. S Something like that, yeah. Or whatever would be technologically reasonable. 'Cause it could b it could it could be that f it could be that functionally that doesn't make it any better, but that just the appeal of of not having You know, these days there's a r pe things in people's homes are becoming more and more like chic, you know. Um, nicer materials and might be be worth exploring anyway. Okay. Um. Before we wrap up, just to make sure we're all on the same page here, um, do we We were given sort of an example of a coffee machine or something, right? Well, um are we at ma right now on the assumption that our television remote control may have features which go beyond the television? Or are we keeping sort of like a a design commitment to television features? I I don't know. Yep. Yeah, sure. Okay. Okay, yeah. Okay. Okay. Okay. Alright.
'''
summarizer(text)
```

# Example 4
```python
from transformers import pipeline
summarizer = pipeline("summarization", model="knkarthick/MEETING_SUMMARY")
text = '''
Das : Hi and welcome to the a16z podcast. I’m Das, and in this episode, I talk SaaS go-to-market with David Ulevitch and our newest enterprise general partner Kristina Shen. The first half of the podcast looks at how remote work impacts the SaaS go-to-market and what the smartest founders are doing to survive the current crisis. The second half covers pricing approaches and strategy, including how to think about free versus paid trials and navigating the transition to larger accounts. But we start with why it’s easier to move upmarket than down… and the advantage that gives a SaaS startup against incumbents.
David : If you have a cohort of customers that are paying you $10,000 a year for your product, you’re going to find a customer that self-selects and is willing to pay $100,000 a year. Once you get one of those, your organization will figure out how you sell to, how you satisfy and support, customers at that price point and that size. But it’s really hard for a company that sells up market to move down market, because they’ve already baked in all that expensive, heavy lifting sales motion. And so as you go down market with a lower price point, usually, you can’t actually support it.
Das : Does that mean that it’s easier for a company to do this go-to-market if they’re a new startup as opposed to if they’re a pre-existing SaaS?
Kristina : It’s culturally very, very hard to give a product away for free that you’re already charging for. It feels like you’re eating away at your own potential revenue when you do it. So most people who try it end up pulling back very quickly.
David : This is actually one of the key reasons why the bottoms up SaaS motion is just so competitive, and compelling, and so destructive against the traditional sales-driven test motion. If you have that great product and people are choosing to use it, it’s very hard for somebody with a sales-driven motion, and all the cost that’s loaded into that, to be able to compete against it. There are so many markets where initially, we would look at companies and say, “Oh, well, this couldn’t possibly be bottoms up. It has to be sold to the CIO. It has to be sold to the CSO or the CFO.” But in almost every case we’ve been wrong, and there has been a bottoms up motion. The canonical example is Slack. It’s crazy that Slack is a bottoms up company, because you’re talking about corporate messaging, and how could you ever have a messaging solution that only a few people might be using, that only a team might be using? But now it’s just, “Oh, yeah, some people started using it, and then more people started using it, and then everyone had Slack.”
Kristina : I think another classic example is Dropbox versus Box. Both started as bottoms up businesses, try before you buy. But Box quickly found, “Hey, I’d rather sell to IT.” And Dropbox said, “Hey, we’ve got a great freemium motion going.” And they catalyzed their business around referrals and giving away free storage and shared storage in a way that really helped drive their bottoms up business.
Das : It’s a big leap to go from selling to smaller customers to larger customers. How have you seen SaaS companies know or get the timing right on that? Especially since it does seem like that’s really related to scaling your sales force?
Kristina : Don’t try to go from a 100-person company to a 20,000-person company. Start targeting early adopters, maybe they’re late stage pre-IPO companies, then newly IPO’d companies. Starting in tech tends to be a little bit easier because they tend to be early adopters. Going vertical by vertical can be a great strategy as well. Targeting one customer who might be branded in that space, can help brand yourself in that category. And then all their competitors will also want your product if you do a good job. A lot of times people will dedicate a sales rep to each vertical, so that they become really, really knowledgeable in that space, and also build their own brand and reputation and know who are the right customers to target.
Das : So right now, you’ve got a lot more people working remote. Does this move to remote work mean that on-premise software is dying? And is it accelerating the move to software as a service?
Kristina : This remote work and working from home is only going to catalyze more of the conversion from on-premise over to cloud and SaaS. In general, software spend declines 20% during an economic downturn. This happened in ’08, this happened in ’01. But when we look at the last downturn in ’08, SaaS spend actually, for public companies, increased, on average, 10%, which means there’s a 30% spread, which really shows us that there was a huge catalyst from people moving on-premise to SaaS.
David : And as people work remote, the ability to use SaaS tools is much easier than having to VPN back into your corporate network. We’ve been seeing that, inside sales teams have been doing larger and larger deals, essentially moving up market on the inside, without having to engage with field sales teams. In fact, a lot of the new SaaS companies today rather than building out a field team, they have a hybrid team, where people are working and closing deals on the inside and if they had to go out and meet with a customer, they would do that. But by and large, most of it was happening over the phone, over email, and over videoconferencing. And all the deals now, by definition, are gonna be done remote because people can’t go visit their customers in person.
Das : So with bottoms up, did user behavior and buyer behavior change, so the go-to-market evolved? Or did the go-to-market evolve and then you saw user and buyer behavior change? I’m curious with this move to remote work. Is that going to trigger more changes or has the go-to-market enabled that change in user behavior, even though we see that change coming because of a lot of forces outside of the market?
Kristina : I definitely think they are interrelated. But I do think it was a user change that catalyzed everything. We decided that we preferred better software, and we tried a couple products. We were able to purchase off our credit card. And then IT and procurement eventually said, “Wow, everyone’s buying these already, I might as well get a company license and a company deal so I’m not paying as much.” While obviously software vendors had to offer the products that could be self-served, users started to realize they had the power, they wanted to use better software, they paid with their credit cards. And now software vendors are forced to change their go-to-market to actually suit that use case.
Das : If that’s the case that when user behavior has changed, it’s tended to be the catalyzing force of bigger changes in the go-to-market, what are some of the changes you foresee for SaaS because the world has changed to this new reality of remote work and more distributed teams?
David : We’re in a very uncertain economic environment right now. And a couple of things will become very clear over the next 3 to 9 to 15 months — you’re going to find out which SaaS products are absolutely essential to helping a business operate and run, and which ones were just nice to have and may not get renewed. I think on the customer, buying side, you’re very likely to see people push back on big annual commitments and prefer to go month-to-month where they can. Or you’ll see more incentives from SaaS startups to offer discounts for annual contracts. You’re going to see people that might sign an annual contract, but they may not want to pay upfront. They may prefer to meter the cash out ratably over the term of the contract. And as companies had empowered and allowed budget authority to be pushed down in organizations, you’re gonna see that budget authority get pulled back, more scrutiny on spending, and likely a lot of SaaS products not get renewed that turned out to not be essential.
Kristina : I think the smartest founders are making sure they have the runway to continue to exist. And they’re doing that in a couple of ways. They’re preserving cash, and they are making sure that their existing customers are super, super happy, because retaining your customers is so important in this environment. And they’re making sure that they have efficient or profitable customer acquisition. Don’t spend valuable dollars acquiring customers. But acquire customers efficiently that will add to a great existing customer base.
Das : To go into pricing and packaging for SaaS for a moment, what are some of the different pricing approaches that you see SaaS companies taking?
Kristina : The old school way of doing SaaS go-to-market is bundle everything together, make the pricing super complex, so you don’t actually understand what you’re paying for. You’re forced to purchase it because you need one component of the product. New modern SaaS pricing is keep it simple, keep it tied to value, and make sure you’re solving one thing really, really well.
David : You want to make it easy for your customers to give you money. And if your customers don’t understand your pricing, that’s a huge red flag. Sometimes founders will try to over engineer their pricing model.
Kristina : We talk a lot about everything has to be 10X better than the alternatives. But it’s much easier to be 10X better when you solve one thing very, very well, and then have simple pricing around it. I think the most common that most people know about is PEPM or per employee per month, where you’re charging basically for every single seat. Another really common model is the freemium model. So, think about a Dropbox, or an Asana, or a Skype, where it’s trigger based. You try the product for free, but when you hit a certain amount of storage, or a certain amount of users, then it converts over to paid. And then you also have a time trial, where you get the full experience of the product for some limited time period. And then you’re asked if you want to continue using the product to pay. And then there’s pay as go, and particularly, pay as you go as a usage model. So, Slack will say, “Hey, if your users aren’t actually using the product this month, we won’t actually charge you for it.”
David : The example that Kristina made about Slack and users, everybody understands what a user is, and if they’re using the product, they pay for it, and if they’re not using it, they don’t pay for it. That’s a very friendly way to make it easy for your customers to give you money. If Slack came up with a pricing model that was like based on number of messages, or number of API integration calls, the customer would have no idea what that means.
Kristina : There’s also the consumption model. So Twilio only charges you for every SMS text or phone call that you make on the platform any given month. And so they make money or lose money as your usage goes. The pricing is very aligned to your productivity.
David : Generally, those are for products where the usage only goes in one direction. If you think of a company like Databricks, where they’re charging for storage, or Amazon’s S3 service, it is very aligned with the customer, but it also strategically aligns with the business because they know the switching cost is very high, the churn is very low. And generally, in those businesses, you’re only going to store more data, so they can charge based on usage or volume of data.
Kristina : Recently, there’s been a huge trend of payment as a revenue. It’s particularly common in vertical markets where SaaS companies are adding payments as a revenue in addition to their employee or subscription revenue. If you look at Shopify, for example, more than 50% of their revenue is actually payment revenue. They’re making money every single time you purchase something off one of their shopping cart websites.
Das : When you’re working with a founder or a SaaS startup, how have you seen them find the right pricing model for their product, for their market?
Kristina : Step one is just talk to a lot of customers. Try to figure out what is the market pricing for possible alternatives or competitors, understand their pain points and their willingness to pay. And just throw a price out there, because you have to have a starting point in order to actually test and iterate. Particularly in the SMB, or the bottoms up business, you can test and iterate pretty quickly because you have so many data points.
David : I always tell founders, step one is to just go out there and talk to customers. Step two is just double your prices. I don’t think there’s ever been a great company with a great product that’s fallen apart because their pricing was wrong. But a lot of SaaS startup founders really under price, and you don’t want to find out two or three years later that you were 200% underpriced. A very common thing that SaaS companies do, they’ll have the basic package that either is free or low cost, that you can just sign up online for. They’ll have a middle package where they share some pricing, and then they’ll have the enterprise package where you have to contact sales to find out more. And that way they don’t actually have to show the pricing for that third package. And that gives the salespeople the flexibility to adjust pricing on a per deal basis.
Das : When you’re working with companies, why are they underpricing their products?
David : I think it’s psychological. People need to price on value, and they don’t know how much value they’re delivering relative to “Oh, it only cost me $100 a month to provide this service, so I just need to charge $200.” But if it turns out you’re saving your customer $50,000 a year, then you’re wildly underpriced. You have to remember that SaaS is essentially a proxy for outsourced IT. You’re spending money on a SaaS service to not pay to develop something internally, or to have to pay IT to support something that’s more complex on-prem. Software is much cheaper than people, and so generally, the price point can be much higher.
Kristina : And the other thing is your value increases over time. You’re delivering more features, more products, you understand the customer better. It’s the beauty of the SaaS model and cloud model that you can iterate and push code immediately, and the customer immediately sees value. A lot of times people have the same price point from the first customer sold to three years later and the 200th customer. Quite frankly, you’ve delivered so much value along the way that your price point should have gone up. The other thing I’ll say is a lot of people discount per seat pricing a lot as they move up market. We tend to tell people that the best validation of your product having great product market fit is your ability to hold your price point. So while there is some natural discounting on a per seat basis because people do deserve some volume discounting, I would say try to resist that as much as possible.
Das : Especially for a technical founder, it’s so tempting to get in there and fiddle with these knobs. How do you know when it is time to experiment with your pricing and packaging?
David : If you’re looking at your business and you see that you are doing more deals, and they’re closing faster, you should raise your pricing. And you pay attention to how long it takes to close deals and whether the number of deals is staying consistent as you do that. And, at some point, you’re going to find out when you’re losing deals on price. I think a moment where companies have to plan ahead to avoid having to course correct is after they roll out massive pricing and packaging changes, which are pretty natural as companies move up market. But how they navigate that transition to larger accounts, and how they either bring along or move away from those smaller, earlier customers who got them to where they are, tends to be really important because they can get a lot of noise on Twitter, they can get a lot of blowback from their customers. So Zendesk is a company where they rolled out a major packaging change. And when they rolled it out, they hadn’t planned on grandfathering in their early customers. They got a lot of pushback, and very quickly, they put out a blog post and said, “We hear what you’re saying, we appreciate you building the business that we’ve become today. We do need to have a package for the future. But all the people that have been customers so far will be grandfathered in for at least a period of time into the old model.”
Kristina : If you iterate pricing constantly, you don’t really have this problem because your customers will be used to pricing changes. You normally pair them with new features, and it all kind of works out. But if you have to go through a big grandfather change, I tend to lean towards treating your early customers really, really well. They adopted when you weren’t a big company yet. They probably co-built the product with you in many ways. And so, it’s great to get more dollars out of your customer base, but treat your early customers well.
Das : Are there any other failure modes that you see startups really falling into around pricing and packaging or any common mistakes that they make?
David : I think a lot of founders don’t always map out the cost or model of their pricing and their product relative to their cost of actually doing sales and marketing and customer acquisition.
Kristina : Inside sales is so popular in Silicon Valley. When you’re selling more to an SMB or mid-market type customer, the expectation is that you’re educating and helping the prospective customer over the phone. And so, you’re not expected to be as high touch. But 5K is almost the minimum price point you need to sell to the SMB with an inside sales team in order to pay for the outbound costs and all the conversions, because there is typically a team that sits around the quota carrying rep. And so, price matching — how much your price point is compared to what your go-to-market motion is — matters a lot. Other big failure modes that I see, people guess the ramp time of a sales rep wrong. And ramp time really ties to the segment of customer you’re selling into. It tends be that if you’re selling into the enterprise, the ramp time for sales reps, because sales cycles are so long, tend to be much longer as well. They could be six months plus, could be a year. While if you’re selling more into SMB or mid-market, the ramp time to get a rep up and running can be much shorter, three to six months. Because the sales cycles are shorter, they just iterate much faster, and they ramp up much more quickly.
David : The other thing that people have to understand is that sales velocity is a really important component to figuring out how many reps you should be hiring, whether they should be inside reps or field reps. If it takes you 90 days to close a deal, that can’t be a $5,000 a year deal, that has to be a $50,000 or even $150,000 a year deal.
Das : Kristina, I know you’ve done a lot of work with metrics. So how do those play in?
Kristina : Probably the one way to sum it all together is how many months does it take to pay back customer acquisition cost. Very commonly within the SaaS world, we talk about a 12-month CAC payback. We typically want to see for every dollar you spend on sales and marketing, you get a dollar back within a year. That means you can tweak the inputs any way you want. Let’s say that doing paid acquisition is really effective for you. Then, you can spend proportionally more on paid acquisition and less on sales reps. Vice versa, if you have a great inbound engine, you actually can hire a lot more sales reps and spend more on sales headcount. With all formulas, it’s a guide rail, so if you have customers that retain really, really well, let’s say you’re selling to the enterprise, and you’ve got a 90% or 95% annual retention rate, then your CAC payback could be between 12 and 24 months. But let’s say you’re selling to the SMB and churn is 2% or 3% monthly, which ends up being like 80% to 90% annual retention. Then, because your customer is less sticky, I would recommend looking at a CAC payback of 6 to 12 months.
Das : How should you think about doing a free trial versus a paid trial?
David : On the one hand, the bottoms up motion where people can try essentially a full version of a product before they buy it is extremely powerful. On the other hand, I’ve started to try to think about how I advise companies, when they are thinking about a free trial for something that might cost $100,000 or $200,000 a year? Do we do a paid pilot that has some sort of contractual obligation that if we meet then turns into a commercial engagement?
Kristina : I do think the beauty of the bottoms up business is that you can get people to try the entire experience of the product for free, and they fall in love with it, and a certain percentage will convert. And that works really, really well for products that can self-serve. When you start moving up market to more complex products, the challenge with trials is it takes work to actually implement the product, whether it be integrations, IT has to give access, etc. You lose that self-serve ability, which is so amazing in the trial. And so, I tend to be more in the camp of paid trials, if it costs you money to actually deploy the trial. And when you’re selling to bigger customers, they associate value when they have to pay. Once a customer has to pay you, then they feel a need to make the project successful and thus they will onboard, schedule things, give you data and access.
David : If you can get to a point where you get the customer to do that paid pilot, such that the only difference between a pilot and an actual customer is just the signing of a contract, that’s very powerful. Now, that does force you to have a really good pre-sales motion to make sure that you can deliver on the promise you’ve made your customers. When companies don’t have a great product, and they paper over it with professional services and sales engineering and post-sales support, that paid pilot thing doesn’t work because the experience isn’t good enough. So, it really is incumbent on the SaaS company that does a paid pilot to make sure that they are able to deliver on that experience.
Kristina : And one emerging trend recently is people signing an annual contract with a one or three month out, as a replacement to the paid pilot. Because it’s the best of both worlds, the SaaS company that’s selling the product gets a higher level of commitment. And the customer gets the optionality of opting out in the same way as a trial without any clawback. It really comes down to where procurement falls. Sometimes procurement is at the beginning of that decision, which makes it more like an annual contract. Sometimes procurement is at the one or three month opt-out period, which means the customer already has a great experience, loves the product, and it is an easier way to convert procurements to actually sign on…
David : And that is a really good segue into renewals. I always tell founders, you might have this subscription business, but it’s not a recurring revenue business until the second year when the revenue actually recurs. I think you really have the first three months to get a customer up and running and happy. And if they’re not, you then have about three months to fix it. And if all that works out, then the remaining six months of the contract can be focused on upsell and expansion.
Das : Awesome. Thank you, Kristina. Thank you, David.
Kristina : Thanks so much for having us. This was fun.
David : Yeah, a lot of fun, great topics, and our favorite thing to talk about.
'''
summarizer(text)
```