What asymmetric sigmoid function defined on [0, ∞[ could I use to model my data?

2.7k Views Asked by At

I need to find a model to fit my data, defined on [0, ∞[ or even on ]0, ∞[. It seems that some kind of (reversed) sigmoid function would work, but my data aren't symmetric, so traditional sigmoid functions don't really work.

I tried tweaking a Gomertz function, but as you can see below, it decreases too slowly at first, and then too fast. Any ideas?

My data points in purple, a tweaked Gomertz function in green

Same thing, but with the X axis on a log scale

And here, the raw data:

x   y
0.3535533906    9.9979731306
0.5 9.9979441096
0.7071067812    9.997896157
0.7071067812    9.9978749407
1   9.997780566
1   9.9977588907
1.2247448714    9.9976603783
1.4142135624    9.9975407811
1.4142135624    9.9975199294
1.4142135624    9.9975177632
1.5811388301    9.9973989145
1.7320508076    9.9972495691
2   9.997075222
2   9.9970653382
2   9.9970071052
2.2360679775    9.9968735327
2.4494897428    9.9965729325
2.4494897428    9.9965735888
2.8284271247    9.9963045626
2.8284271247    9.9961509644
2.8284271247    9.996102247
2.8284271247    9.9962285165
3.1622776602    9.9956701877
3.1622776602    9.9956411184
3.4641016151    9.9950417481
3.4641016151    9.9952492462
3.7416573868    9.9946102874
4   9.9942744597
4   9.9943302815
4   9.9939916438
4   9.9943702025
4.472135955 9.9931768043
4.472135955 9.9932582478
4.8989794856    9.9924335386
4.8989794856    9.9919089218
4.8989794856    9.9927108557
5.2915026221    9.9914500289
5.6568542495    9.9913424021
5.6568542495    9.9900219326
5.6568542495    9.9907046142
5.6568542495    9.9898771885
5.6568542495    9.9904639461
6.3245553203    9.9879377579
6.3245553203    9.9879743157
6.3245553203    9.9897649623
6.9282032303    9.9851938132
6.9282032303    9.9868618753
6.9282032303    9.9877988638
7.4833147735    9.9849382687
7.4833147735    9.9859169334
8   9.9830432438
8   9.9830101258
8   9.9826662298
8   9.9827933477
8   9.982817833
8.94427191  9.9792529317
8.94427191  9.9802033408
8.94427191  9.9823864194
9.7979589711    9.9747179652
9.7979589711    9.9749611618
9.7979589711    9.9760116806
9.7979589711    9.9776693712
10.5830052443   9.9714823251
11.313708499    9.9686710997
11.313708499    9.9677408886
11.313708499    9.9661118054
11.313708499    9.9679970902
11.313708499    9.9680414586
11.313708499    9.9664268208
12.6491106407   9.9591613419
12.6491106407   9.9609296247
12.6491106407   9.9626980052
13.8564064606   9.9521806585
13.8564064606   9.9526334571
13.8564064606   9.9558756549
14.9666295471   9.9446790369
14.9666295471   9.9498590399
16  9.9445549856
16  9.9330806462
16  9.9367293948
16  9.9413555985
16  9.9401807259
16.9705627485   9.9346889776
17.88854382 9.9196265421
17.88854382 9.9253827006
17.88854382 9.9370654524
19.5959179423   9.9100270227
19.5959179423   9.9041481786
19.5959179423   9.913014108
19.5959179423   9.919521431
21.1660104885   9.8947578693
22.627416998    9.8918094719
22.627416998    9.8713650885
22.627416998    9.8733293734
22.627416998    9.8798409365
22.627416998    9.8777822652
22.627416998    9.8715468945
24  9.8707318867
25.2982212813   9.8382348743
25.2982212813   9.841944607
25.2982212813   9.8685173357
27.7128129211   9.8196631997
27.7128129211   9.8234759383
27.7128129211   9.8308509026
28.2842712475   9.824205574
29.9332590942   9.7912290073
29.9332590942   9.7981835254
32  9.7689532343
32  9.7678748253
32  9.7545795668
32  9.758628004
32  9.7655441336
33.941125497    9.7415241942
33.941125497    9.7222751527
35.77708764 9.7071517554
35.77708764 9.7131108475
35.77708764 9.7596412415
39.1918358845   9.636118575
39.1918358845   9.6217432379
39.1918358845   9.6325205631
39.1918358845   9.6792384173
39.5979797464   9.6866221408
40  9.6450847668
42.332020977    9.5850964021
45.2548339959   9.5794257293
45.2548339959   9.5220969239
45.2548339959   9.5387735428
45.2548339959   9.5203534668
45.2548339959   9.5485919631
45.2548339959   9.5483055814
48  9.499226795
48  9.4408029209
50.5964425627   9.4072535945
50.5964425627   9.4027464987
50.5964425627   9.4676509042
50.9116882454   9.4323128908
55.4256258422   9.3174138587
55.4256258422   9.3008138587
55.4256258422   9.3999171698
56  9.3143904143
56.5685424949   9.3465622319
56.5685424949   9.271269093
58.7877538268   9.2476393817
59.8665181884   9.2211094876
59.8665181884   9.2274917118
64  9.132537622
64  9.107278809
64  9.0319794364
64  9.1088696748
64  9.0975565848
67.8822509939   9.0844153404
67.8822509939   8.9932419602
67.8822509939   8.98866032
71.55417528 8.8683766313
71.55417528 9.0182329381
71.55417528 9.0916227983
72  8.9556913491
75.894663844    8.7364886005
78.3836717691   8.7381626778
78.3836717691   8.7183828582
78.3836717691   8.7196318764
78.3836717691   8.8337361779
79.1959594929   8.7894671034
79.1959594929   8.7577623117
80  8.6736134642
80  8.655709055
83.1384387633   8.5578709673
84.6640419541   8.5544338027
90.5096679919   8.5064435198
90.5096679919   8.3747730414
90.5096679919   8.4634959724
90.5096679919   8.3260757554
90.5096679919   8.4511242246
90.5096679919   8.4194825858
96  8.2575212242
96  8.1257670009
96  8.1453620994
97.9795897113   8.0710679496
101.1928851254  8.0791459917
101.1928851254  8.1235050394
101.1928851254  8.1687117775
101.8233764909  8.0734844887
101.8233764909  8.0099313959
107.33126292    7.894393358
110.8512516844  7.8206052335
110.8512516844  7.7661708866
110.8512516844  7.9481519641
112 7.8145896471
112 7.5834543001
113.1370849898  7.7650979962
113.1370849898  7.6457065055
113.1370849898  7.747631427
117.5755076536  7.4402250135
117.5755076536  7.5244375513
119.7330363768  7.4775244535
119.7330363768  7.6038757053
126.4911064067  7.2773311199
127.2792206136  7.3919916031
128 7.3024457228
128 7.2317346782
128 7.3206480776
128 7.3325210272
128 7.3311883257
135.7645019878  7.1130947152
135.7645019878  7.1199026643
135.7645019878  7.1155703141
137.1714255959  6.9092688699
138.5640646055  7.0195246551
143.10835056    6.8011022721
143.10835056    6.9297952756
143.10835056    7.365793704
144 6.8812037556
144 6.6660381104
151.7893276881  6.560975023
151.7893276881  6.6885403071
156.7673435381  6.5327004621
156.7673435381  6.6007055522
156.7673435381  6.5953251087
156.7673435381  6.8568337688
158.3919189858  6.3622964876
158.3919189858  6.4622263028
160 6.447826802
160 6.3981646505
160 6.5292012988
166.2768775266  6.3970888969
166.2768775266  6.2014574798
169.3280839081  6.1050638233
176.3632614804  5.9712312801
177.0875489694  6.1574205433
178.8854382 6.1642171555
179.5995545652  6.0474491972
180 6.0945833864
181.0193359838  5.9120240244
181.0193359838  5.9804016409
181.0193359838  6.0511108883
181.0193359838  6.064449925
181.0193359838  6.1010720394
192 5.6730484564
192 5.8679729754
192 5.762090217
193.9896904477  5.7328303265
195.9591794227  5.6865393773
195.9591794227  5.8414142213
202.3857702508  5.4642097534
202.3857702508  5.6691623288
202.3857702508  5.6746554987
203.6467529817  5.4539929285
203.6467529817  5.4719954869
214.66252584    5.4614359928
214.66252584    5.4081565789
221.7025033688  5.1537993618
221.7025033688  5.3406438594
221.7025033688  5.4448354378
224 5.1245688973
224 5.1610529142
226.2741699797  4.9202500719
226.2741699797  5.0484825444
226.2741699797  5.2734249456
227.6839915321  5.1231905701
235.1510153072  4.8549393009
235.1510153072  5.0841299276
235.1510153072  5.138479726
239.4660727535  4.8763407691
239.4660727535  5.2238402659
249.4153162899  4.7706038039
250.43961348    4.8043270676
252.9822128135  4.7634355318
252.9822128135  4.8730618655
253.9921258622  4.8227058213
254.5584412272  4.660655104
256 4.6112695833
256 4.5680305804
256 4.720787087
256 4.7141155513
271.5290039756  4.3850359182
271.5290039756  4.5556890129
271.5290039756  4.5836248516
274.3428511917  4.3684198575
274.3428511917  4.4313833041
277.128129211   4.3556239608
277.128129211   4.453772521
286.21670112    4.2906048634
286.21670112    4.3571147164
286.21670112    4.7842136283
288 4.0841849686
288 4.2385242942
299.3325909419  4.1786581361
303.5786553762  4.1126142057
303.5786553762  4.1547024841
303.5786553762  4.3397497762
313.5346870762  3.8955234188
313.5346870762  4.0234591649
313.5346870762  4.1587333897
313.5346870762  4.2087621254
316.7838379716  3.8795791997
316.7838379716  4.0138808577
320 3.8170141118
320 3.9565940992
320 3.9662051391
321.99378876    3.8786050799
332.5537550532  3.7148010655
332.5537550532  3.8518873535
332.5537550532  4.0904649704
338.6561678163  3.8016331307
352.7265229608  3.5635876014
352.7265229608  3.6414218734
354.1750979389  3.6089575357
354.1750979389  3.6522994515
357.7708764 3.5375309981
357.7708764 3.6945049419
359.1991091303  3.5796275351
359.1991091303  3.8227901649
360 3.4870982923
362.0386719675  3.4554595685
362.0386719675  3.5489674563
362.0386719675  3.6207772488
362.0386719675  3.6337456513
384 3.3439653906
384 3.4568265624
384 3.4628743885
387.9793808954  3.282574716
387.9793808954  3.3718524427
391.9183588453  3.2001610965
391.9183588453  3.2934972883
391.9183588453  3.4342471559
404.7715405016  3.1728359399
404.7715405016  3.304873072
404.7715405016  3.5236283741
407.2935059635  3.1079261982
407.2935059635  3.2755188107
419.0656273187  3.161646244
423.3202097703  3.1804540486
429.32505168    3.1080120179
429.32505168    3.1849101905
429.32505168    3.4635793732
440.908153701   2.9023848255
443.4050067376  2.9297611292
443.4050067376  2.9859094462
443.4050067376  3.3023894256
448 2.9272162504
448 2.9515935151
452.5483399594  2.8407194486
452.5483399594  2.9556381324
452.5483399594  3.0266921549
455.3679830642  2.8706210835
455.3679830642  2.9277916535
470.3020306144  2.8192585
470.3020306144  2.982599651
470.3020306144  2.9960400484
478.9321455071  2.8282043474
478.9321455071  2.9468040236
498.8306325798  2.583330307
498.8306325798  2.7150682525
500.87922696    2.6421950218
500.87922696    2.7472335526
505.9644256269  2.5681808833
505.9644256269  2.7266048526
505.9644256269  2.8104109804
507.9842517244  2.7201572269
509.1168824543  2.5065140145
512 2.5975501313
512 2.7098815111
512 2.6817203113
538.7986636954  2.5237781107
543.0580079513  2.5343618849
543.0580079513  2.5750386873
543.0580079513  2.5668231833
548.6857023834  2.4456474015
548.6857023834  2.5491671602
554.256258422   2.3726006443
554.256258422   2.5548411469
554.256258422   2.644359126
569.2099788303  2.3023667598
572.4334022399  2.3309658451
572.4334022399  2.4247371678
572.4334022399  2.7039728753
576 2.2580698069
576 2.4153096858
592.6482936785  2.3208210481
598.6651818838  2.2749079564
598.6651818838  2.5122432977
607.1573107523  2.2190148625
607.1573107523  2.4296776349
623.5382907248  2.1056746805
627.0693741525  2.1436859935
627.0693741525  2.2831621684
627.0693741525  2.4179178085
633.5676759431  2.1270503046
633.5676759431  2.2656390326
640 2.0903131155
640 2.1962743929
640 2.2423850538
643.9875775199  2.1097691369
643.9875775199  2.2623700637
665.1075101064  2.0588993183
665.1075101064  2.2420718896
677.3123356325  2.0460207101
705.4530459216  1.880558301
705.4530459216  2.0130708853
708.3501958777  1.9783303356
708.3501958777  2.0765191586
715.5417527999  1.882917264
715.5417527999  1.9572238391
715.5417527999  2.1686078382
718.3982182606  1.9304284455
718.3982182606  2.0574438918
720 1.8211324085
724.077343935   1.8658839637
724.077343935   1.9507150757
724.077343935   1.9608401489
761.9763775866  1.8190304262
768 1.8226705713
768 1.8504945376
775.9587617909  1.7811478666
775.9587617909  1.9556578857
783.8367176906  1.7090958485
783.8367176906  1.847196316
783.8367176906  1.9663014797
804.9844718999  1.6748031876
809.5430810031  1.6952134318
809.5430810031  1.899465663
814.5870119269  1.693791842
814.5870119269  1.7622703502
838.1312546374  1.6521199758
838.1312546374  1.780678696
846.6404195407  1.6701361319
858.6501033599  1.679250853
858.6501033599  1.9068672069
881.8163074019  1.5666945056
886.8100134753  1.5429750314
886.8100134753  1.72760035
896 1.5631217533
896 1.5881359322
905.0966799188  1.5158111145
905.0966799188  1.6250113719
905.0966799188  1.5842557476
910.7359661285  1.5187575511
910.7359661285  1.6619700548
940.6040612287  1.5546374695
940.6040612287  1.6254435014
957.8642910141  1.4636121055
957.8642910141  1.5764887253
997.6612651597  1.396259956
997.6612651597  1.5635128654
1001.7584539199 1.4512842752
1001.7584539199 1.602981421
1011.9288512539 1.3387369187
1011.9288512539 1.5211357437
1015.9685034488 1.3863161926
1018.2337649086 1.3383220729
1024    1.3674719961
1024    1.4278590565
1077.5973273909 1.2668426382
1077.5973273909 1.4178393335
1086.1160159025 1.3235199607
1086.1160159025 1.3335504125
1097.3714047669 1.3455132893
1097.3714047669 1.3925736158
1108.5125168441 1.2386664624
1108.5125168441 1.3902026429
1138.4199576606 1.2034695051
1144.8668044799 1.2268908299
1144.8668044799 1.4492429459
1152    1.2161496436
1152    1.2777845569
1185.2965873569 1.202895207
1197.3303637677 1.143099124
1197.3303637677 1.2793499227
1214.3146215047 1.2539626986
1247.0765814496 1.080295926
1254.138748305  1.152221543
1254.138748305  1.2513988127
1267.1353518863 1.1503902919
1267.1353518863 1.1484875806
1280    1.0856735053
1280    1.1472593418
1287.9751550399 1.1045986251
1287.9751550399 1.2590181328
1330.2150202129 1.1481608168
1346.9966592386 1.0040614018
1354.6246712651 1.0421900625
1410.9060918431 1.016839823
1410.9060918431 1.0857259512
1416.7003917554 1.0446198173
1431.0835055999 0.99053285
1431.0835055999 1.1466897847
1436.7964365212 1.0453234124
1440    0.9701502202
1448.1546878701 0.9876410138
1448.1546878701 1.0379661972
1523.9527551732 0.9192256712
1536    0.9539307498
1551.9175235817 0.9882196467
1567.6734353812 0.906497227
1567.6734353812 0.9781804824
1609.9689437999 0.8654395739
1619.0861620062 0.9386227963
1629.1740238538 0.8730639496
1629.1740238538 0.9163330887
1676.2625092747 0.9028328578
1693.2808390813 0.8190053791
1717.3002067198 0.9589167976
1763.6326148039 0.7952151809
1773.6200269505 0.8751646914
1792    0.835188681
1810.1933598376 0.7883994945
1810.1933598376 0.8235896112
1821.471932257  0.8201411345
1881.2081224575 0.8184385225
1904.9409439665 0.7282825012
1915.7285820283 0.7945652559
1995.3225303194 0.7721619479
2003.5169078398 0.7991285571
2023.8577025078 0.719840126
2036.4675298173 0.6948938405
2048    0.7084098996
2155.1946547818 0.6940733696
2172.2320318051 0.6927335265
2194.7428095337 0.7006571374
2217.0250336882 0.6972557819
2276.8399153212 0.6512216718
2289.7336089598 0.7022353115
2304    0.6138353326
2394.6607275353 0.6170132713
2494.1531628992 0.610297418
2508.27749661   0.6073264722
2534.2707037726 0.5730046272
2560    0.5513206451
2575.9503100798 0.6239501259
2693.9933184772 0.539733654
2821.8121836862 0.5396575193
2862.1670111997 0.5586966342
2880    0.4944326606
2896.3093757401 0.4923064599
3135.3468707625 0.4714507449
3219.9378875997 0.4931032241
3258.3480477076 0.4435256704
3527.2652296078 0.4241201511
3620.3867196751 0.3924015328
4072.9350596345 0.346517912
1

There are 1 best solutions below

12
On BEST ANSWER

I would suggest plotting the data on a $\log$-$\log$ scale. I've done so, and that high end tail is definitely a power law (straight line on a $\log$-$\log$ scale). You can't use a straight Pareto distribution, though, because of the curvature at the low end.

If you don't particularly care about the theoretical content, you could always throw a smoothly broken power law at it. Given a positive function, $f(x)$, you can add a break to it around $x_n$ by multiplying it by: $$B(x, s, \beta, x_n) = \left(1 + \left[\frac{x}{x_n}\right]^{|\beta|s}\right)^{\operatorname{sgn}(\beta)/s}.$$ This will add a bend to the function around $x_n$ with a width controlled by $s$ (the sharpness - the larger $s$ the sharper the bend, with $s\rightarrow \infty$ leading to a corner). If $f$ is a power law and $\lim_{x\rightarrow 0} \frac{\operatorname{d} \ln [f(x)\, B(x)]}{\operatorname{d} \ln x} = m$, then $\lim_{x\rightarrow \infty} \frac{\operatorname{d} \ln [f(x)\, B(x)]}{\operatorname{d} \ln x} = m + \beta$.

For this sigmoid the low $x$ behavior is obviously $F(x) = \mathrm{const}$, but I don't know if you'll need one break or two to model it's behavior.

Note that unless there's something interesting about the second derivative of this function, this process of throwing breaks at it won't reveal anything particularly interesting. That's because if you look at $B$ in $\log$-$\log$ space then as $s\rightarrow \infty$, $\log B(x) \rightarrow \beta R(\log (x/x_n))$, with $R$ the ramp function. Because the ramp function is a Green's function for the second derivative operator, any twice differentiable map from $\mathbb{R}^+ \rightarrow \mathbb{R}^+$ can be modeled arbitrarily well if you use enough breaks with large enough sharpness.

Looking at the CDF ($=1 - \mathrm{SF}$) your data at the low end is approximately: $$F(x \ll 1) \approx 9.9979731306 - 2\times 10^{-4} x^2 + \ldots $$ At the high end, it's approximately: $$F(x \gg 1000) \approx \frac{1.5\times 10^3}{x}.$$

Looking at a graph where I tweaked the parameters using "$\chi$ by eye", a broken power law with $\beta = -1$, $s=2$, $x_n=150$, and normalization of $9.9979731306$ does an okay job. That is: $$F(x) \approx 9.9979731306 \left(1 + \left[\frac{x}{150}\right]^2\right)^{-1/2}.$$ You could, of course, achieve better by actually fitting the curve using an optimization package.